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Foreword 


This  volume  is  a  background  paper  for  OTA's  assessment,  The  Implications  of 
Cost-Effectiveness  Analysis  of  Medical  Technology.  That  assessment  analyzes  the 
feasibility,  implications,  and  usefulness  of  applying  cost-effectiveness  and  cost-benefit 
analysis  (CEA/CBA)  in  health  care  decisionmaking.  The  major,  policy-oriented  report 
of  the  assessment  was  published  in  August  1980.  In  addition  to  the  main  report  and 
this  background  report,  there  will  be  four  other  background  papers:  1)  a  document 
which  addresses  methodological  issues  and  reviews  the  CEA/CBA  literature,  2)  a  diag- 
nostic X-ray  case  study,  3)  17  other  case  studies  of  individual  medical  technologies, 
and  4)  a  review  of  international  experience  in  managing  medical  technology.  Another 
related  report  was  published  in  September  of  1979:  A  Review  of  Selected  Federal  Vac- 
cine and  Immunization  Policies. 
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1. 

Summary 


The  treatment  of  mental,  emotional,  and  be- 
havioral dysfunctions  has  become  one  of  the 
most  controversial  areas  of  health  policy.  Even 
though  the  prevalence  and  the  pernicious  effects 
of  mental  disorders  are  well  known  and  have 
been  documented  recently  in  sources  such  as  the 
report  of  the  President's  Commission  on  Mental 
Health  (219),  opinion  about  what  should  be 
done  to  treat  these  problems  is  not  unanimous. 
A  number  of  proposals  to  expand  the  mental 
health  services  system  and  to  make  treatment 
more  widely  available  (e.g.,  through  expanded 
insurance  coverage)  have  been  made.  Yet,  there 
is  not  agreement  about  how  to  expand  mental 
health  services,  nor  about  what  would  be  gained 
by  their  expansion. 

In  light  of  these  disagreements,  it  is  perhaps 
understandable  that  policymakers  have  been  re- 
luctant to  commit  additional  public  resources  to 
mental  health  treatments.  To  make  psychother- 
apeutic treatments  more  widely  available  is  po- 
tentially expensive.  It  has  been  suggested,  how- 
ever, that  the  provision  of  psychotherapy  re- 
duces other  costs  currently  borne  by  society.  A 
central  question  is  the  extent  to  which  psycho- 
therapy can  be  scientifically  assessed  and  its 
value  demonstrated  in  a  way  useful  for  policy- 
making. The  present  report  analyzes  the  current 
scientific  literature  on  the  evaluation  of  psycho- 
therapy. It  examines  the  efficacy  and  cost  effec- 
tiveness of  psychotherapeutic  treatments  and 
considers  both  the  methodological  problems  of 
assessing  psychotherapy  and  the  state  of  current 
knowledge  about  its  effects.  It  is  hoped  that  this 
report  will  inform  the  developing  congressional 
debate  on  Federal  research  and  funding  for 
psychotherapy. 

As  a  background  paper  prepared  in  conjunc- 
tion with  OTA's  assessment  The  Implications  of 
Cost-Effectiveness  Analysis  of  Medical  Technol- 
ogy (see  203),  the  present  report  emphasizes 
methodological  issues  related  to  the  assessment 
of  psychotherapy  and  the  use  of  cost-effective- 
ness/cost-benefit analysis  (CEA/CBA)  to  eval- 
uate psychotherapy's  worth.  It  documents  the 


use  of  various  evaluation  strategies  and  consid- 
ers the  available  evidence  concerning  psycho- 
therapy's  efficacy  and  cost  effectiveness.  The 
conclusions  of  the  report  relate  both  to  the  con- 
duct of  research  on  psychotherapy  and  the 
status  of  present  scientific  knowledge  about 
psychotherapy. 

The  report  discusses  four  issues  centrally  re- 
lated to  the  evaluation  of  psychotherapy:  1)  the 
definition  and  complexity  of  psychotherapy;  2) 
the  degree  to  which  psychotherapy  is  amenable 
to  scientific  analysis  and  the  availability  of  ap- 
propriate methods  for  studying  psychotherapy; 
3)  the  evidence  as  to  psychotherapy's  efficacy, 
including  the  results  of  analyses  that  synthesize 
findings  across  studies;  and  4)  the  appropriate- 
ness of  CEA/CBAs  of  psychotherapy  and  the  re- 
sults of  their  application.  Below,  each  of  these 
sections  of  the  report  is  briefly  summarized. 

Chapter  2  reviews  a  number  of  definitions 
and  views  of  psychotherapy.  It  also  attempts  to 
delimit  the  scope  of  this  background  paper. 
Psychotherapy  is  not  a  simple  treatment,  and 
part  of  the  confusion  about  its  effectiveness  has 
to  do  with  the  use  of  different  views  of  what 
comprises  psychotherapy.  In  order  to  represent 
the  variety  of  contemporary  therapy  practices, 
the  present  report  adopts  a  relatively  compre- 
hensive definition  of  psychotherapy.  Included 
are  treatments  based  on  Freudian  ideas  about 
psychodynamics,  as  well  as  newer  therapies 
based  on  behavioral  theories  of  learning  and 
cognition.  One  finding  of  the  report  is  that 
psychotherapies  are  not  distinguishable  solely 
by  their  theoretical  bases.  In  addition  to  the 
view  of  psychopathology  adopted  by  the  thera- 
pist, therapist  variables  (e.g.,  training  and  per- 
sonality characteristics),  patient  variables  (e.g., 
seriousness  of  condition),  and  the  treatment  set- 
ting (e.g.,  hospital,  private  office)  affect  the  na- 
ture of  psychotherapy  treatments.  Although  the 
inclusion  of  such  complex  factors  makes  the 
analysis  of  psychotherapy  more  difficult,  their 
inclusion  is  necessary  to  adequately  assess  the 
effects  of  psychotherapy. 
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Chapter  3  describes  the  scientific  basis  of  ef- 
forts to  assess  the  effectiveness  of  psychothera- 
py. Although  the  definition  of  psychotherapy 
employed  in  this  report  is  complex,  and  perhaps 
open  to  dispute,  the  methods  for  assessing  psy- 
chotherapy are  better  established.  The  present 
report  describes  and  analyzes  various  methodo- 
logical strategies  for  measuring  the  outcomes  of 
psychotherapeutic  treatment  and  the  ways  in 
which  the  reliability  and  validity  of  these  meas- 
ures are  established.  It  also  describes  the  variety 
of  research  design  strategies  that  have  been  used 
to  establish  cause-effect  relationships  between 
psychotherapy  and  particular  outcomes.  The 
discussion  indicates  the  conditions  under  which 
randomized  control  group  procedures  can  be 
used  to  assess  psychotherapy.  Also  analyzed  are 
the  use  of  quasi-experimental  and  nonexperi- 
mental  procedures  which,  depending  on  what 
types  of  information  are  required,  may  provide 
useful  data.  The  problems  of  carrying  out 
psychotherapy  research  are  also  discussed,  in- 
cluding the  difficulties  of  withholding  treatment 
from  members  of  a  control  group  and  the 
problems  of  assessing  multifactor  treatment 
programs. 

Also  considered  in  chapter  3  are  two  recent 
methodological  developments.  One  is  the  con- 
duct of  program  evaluation  studies,  in  which 
sets  of  psychotherapy  variables  are  investigated 
simultaneously.  Such  evaluation  studies  may  be 
useful  where  several  psychotherapy-related  var- 
iables are  considered  in  conjunction  with  one 
another  and  where  it  is  difficult  to  separate 
treatment  components.  Another  recent  method- 
ological development  is  the  use  of  systematic 
procedures  for  synthesizing  the  findings  of 
multiple  investigations.  These  data  integration 
methods  are  new  and  somewhat  controversial 
procedures  for  assessing  the  implications  of  the 
psychotherapy  research  literature.  Both  of  these 
methods,  along  with  scientifically  rigorous  stud- 
ies of  psychotherapy,  may  prove  useful  to  de- 
veloping mental  health  policy. 

Chapter  4  selectively  describes  the  substan- 
tive literature  on  psychotherapy 's  effects.  A 
number  of  prominent  reviews  of  the  psychother- 
apy literature,  and  the  commentary  generated 
by  these  reviews,  are  analyzed  (along  with  sev- 


eral individual  evaluative  studies).  Despite  some 
fundamental  differences,  both  in  the  criteria  for 
assessing  psychotherapy  and  in  the  studies  in- 
cluded, the  reviews  all  report — under  specified 
conditions — evidence  for  psychotherapy's  effec- 
tiveness. This  finding  is  stronger  the  more  recent 
the  literature  that  is  reviewed.  In  fact,  there 
seems  to  be  litle  negative  evidence  as  to  the  effi- 
cacy of  psychotherapeutic  treatments.  Although 
it  is  difficult  to  make  global  statements,  the  evi- 
dence seems  more  supportive  of  psychotherapy 
than  of  alternative  explanations  (e.g.,  spontane- 
ous remission,  placebo  effects).  The  available 
research,  some  of  which  meets  rigorous  meth- 
odological standards,  seems  to  indicate  that 
psychotherapy  treatment  is  clearly  better  than 
no  treatment.  However,  while  the  literature  sup- 
ports a  generally  positive  conclusion  with  re- 
spect to  the  effectiveness  of  psychotherapy, 
there  is  a  lack  of  specific  information  about  the 
conditions  under  which  psychotherapy  is  effec- 
tive. It  is  not  clear  which  aspects  of  therapy 
(e.g.,  treatment  protocol  v.  the  nature  of  the 
therapist's  relationship  with  the  patient)  are  re- 
sponsible for  particular  outcomes. 

Methods  for  assessing  the  costs  and  benefits 
of  psychotherapy  and  for  developing  CBAs  are 
described  in  chapter  5.  The  application  of  CEA/ 
CBA  to  psychotherapy  is  much  more  recent, 
and  less  developed,  than  efficacy  research.  Al- 
though the  methods  for  CEA/CBAs  of  psycho- 
therapy are  based  on  applications  of  such  analy- 
ses to  other  types  of  health  and  nonhealth  prob- 
lems, in  some  instances  (e.g.,  application  of  the 
"willingness-to-pay"  concept),  the  translation  to 
psychotherapy  is  difficult.  An  additional  prob- 
lem with  psychotherapy  assessments  has  to  do 
with  the  comprehensiveness  of  cost  and  benefit 
assessment — in  particular,  our  ability  to  value 
in  pecuniary  terms  the  effects  of  psychotherapy. 
Because  of  this  difficulty,  much  of  the  recent 
cost  analysis  research  has  involved  cost-effec- 
tiveness comparisons  rather  than  cost-benefit 
comparisons.  Potentially,  however,  both  CEA 
and  CBA  techniques  may  be  useful  to  improve 
our  understanding  of  the  effects  of  psychother- 
apy and  the  resources  necessary  for  its  efficient 
use.  Such  CEA/CBA  research  may  be  an  impor- 
tant adjunct  to  effectiveness  studies  that  will 
enhance  their  policy  use. 
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Chapter  6  reviews  the  available  literature  on 
the  use  of  CEA/CBA  for  asessing  psychothera- 
py. The  literature  which  reports  actual  CEA/ 
CBA  studies  varies,  both  in  terms  of  its  focus  on 
different  problems  and  its  methodological  ade- 
quacy. Because  of  the  problem  of  valuing  psy- 
chotherapy outcomes,  a  great  number  of  the 
available  cost  studies  focus  on  low-cost  treat- 
ments. This  may  create  an  incorrect  impression 
about  psychotherapy's  effects.  In  terms  of  their 
limitation  of  problem  focus,  many  of  these  anal- 
yses have  been  conducted  on  treatment  settings 
characteristics  (e.g.,  institutional  v.  noninstitu- 
tional  care),  rather  than  on  different  therapies 
or  therapists.  The  findings  of  these  studies  seem 
to  indicate  that  more  efficient  ways  of  delivering 
psychotherapy  can  be  developed.  Unfortunate- 
ly, it  is  difficult  to  interpret  the  results  of  many 
of  these  studies,  because  they  ignore  important 
costs  or  benefits  or  because  they  use  inadequate 


research  designs.  In  some  cases  (e.g.,  the  effects 
of  psychotherapy  on  medical  utilization  rates), 
at  least  the  potential  for  psychotherapy  to  pro- 
vide society  with  large  net  benefits  has  been 
demonstrated.  It  will  be  necessary,  however,  to 
validate  these  findings  using  better  procedures 
and  a  wide  range  of  mental  health  problems. 

In  summary,  OTA  finds  that  psychotherapy 
is  a  complex — yet  scientifically  assessable — set 
of  technologies.  It  also  finds  good  evidence  of 
psychotherapy's  positive  effects.  Although  this 
evidence  may  not  be  generalizable  to  the  wide 
range  of  problems  for  which  therapy  is  em- 
ployed, it  suggests  that  additional  research  may 
provide  data  useful  for  the  development  of  men- 
tal health  policy.  Given  the  potential  net  ben- 
efits of  psychotherapy,  this  effort  would  seem  to 
be  justified. 
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The  term  psychotherapy  has  been  used  to  re- 
fer to  a  wide  variety  of  treatments  employed  to 
ameliorate  mental  distress,  mental  illness,  and 
problems  of  coping  with  daily  life.  Psychother- 
apy is  a  global  term,  and  one  commentator  has 
found  at  least  40  different  definitions  in  the 
scientific  literature  (300).  The  definitions  range 
from  broad  and  inclusive  descriptions  of  vari- 
ous ways  of  helping  other  individuals  to  more 
limiting  definitions  which  are  very  specific  as  to 
the  nature  of  the  problem  and  treatment  (see, 
e.g.,  118).  Differences  in  the  definitions  used  by 
those  who  discuss  psychotherapy  are  a  likely 
reason  that  discussions  of  psychotherapy  have 
been  so  difficult  to  understand  and  evaluate. 
The  present  review  attempts  to  specify  the  na- 
ture of  psychotherapy,  so  as  to  avoid  some  of 
these  problems. 

To  illustrate  some  of  the  difficulties  of  differ- 
ent views  of  psychotherapy,  consider  a  broad 
definition  such  as  "the  treatment  of  emotional 
and  personality  problems  and  disorders  by  psy- 
chological means"  (144).  This  definition  refers 
to  treatments  that  may  have  either  a  good  or 
poor  scientific  basis,  as  well  as  to  treatments 
that  are  delivered  by  either  professionally 
trained  or  untrained  personnel.  In  contrast,  a 
more  specific  definition  (which  has  been  favored 
by  some  researchers)  describes  psychotherapy 
in  terms  of  its  techniques,  the  qualifications  of 
the  therapist,  and  nature  of  the  patient's  prob- 
lem (181): 

Psychotherapy  is  taken  to  mean  the  planful 
application  of  techniques  derived  from  estab- 
lished psychological  principles,  by  persons  qual- 
ified through  training  and  experience  to  under- 
stand these  principles  and  to  apply  these  tech- 
niques with  the  intention  of  assisting  individuals 
to  modify  such  personal  characteristics  as  feel- 
ings, values,  attitudes,  and  behaviors  which  are 
judged  by  the  therapist  to  be  maladaptive  or 
maladjustive. 

Apart  from  their  level  of  specificity,  defini- 
tions vary  with  regard  to  what  is  included  as  the 
disorders  to  be  treated  by  psychotherapy.  Al- 
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though  some  definitions,  like  the  above,  are  not 
specific  as  to  the  nature  of  the  disorder  (see, 
e.g.,  84),  diagnosis  is  becoming  more  system- 
atic; viz,  the  newly  developed  Diagnostic  and 
Statistical  Manual  of  Mental  Disorders  (63). 
The  new  diagnostic  manual  includes  disorders 
that  range  from  organic  brain  syndromes,  schiz- 
ophrenic and  paranoid  disorders,  to  affective 
disorders  such  as  depression,  anxiety,  and  disor- 
ders such  as  phobias  or  posttraumatic  stress; 
also  included  are  behavior  aberrations  such  as 
kleptomania  or  pathological  gambling,  person- 
ality disorders,  alcohol-  and  drug-related  prob- 
lems, and  interpersonal  difficulties  such  as  mari- 
tal or  family  disturbances  or  education  and 
work  blockages. 

One  additional  feature  of  the  various  ap- 
proaches to  defining  psychotherapy  is  that  def- 
initions vary  in  terms  of  their  underlying  theo- 
retical assumptions.  These  assumptions  lead  to 
differences  in  defining  the  goal  of  psychothera- 
py, the  basic  techniques,  and  the  role  of  the  pa- 
tient and  therapist.  As  Hogan  (118)  has  noted, 
psychotherapy  can  refer  to  a  wide  variety  of 
vaguely  defined  processes.  Attempts  to  specify 
the  nature  of  psychotherapy  result,  in  part,  in 
definitions  that  vary  in  terms  of  the  practition- 
ers and  settings  which  are  included  as  accept- 
able. Different  definitions  require  different  lev- 
els of  professional  training  and  experience  for 
psychotherapy  practitioners.  For  example,  med- 
ical definitions  of  psychotherapy  (see  163)  may 
limit  primary  responsibility  for  psychotherapy 
treatment  to  physicians,  while  nonmedical  mod- 
els (e.g.,  250)  may  suggest  central  roles  for  psy- 
chologists, social  workers,  and  nurses,  as  well 
as  educational  and  pastoral  counselors.  The 
various  forms  of  psychotherapy  are  further  af- 
fected by  the  variety  of  settings  in  which  ther- 
apy takes  place  (e.g.,  psychiatric  hospitals,  out- 
patient clinics,  private  offices)  and  the  variety  of 
modalities  that  are  offered  (e.g.,  individual, 
family,  or  group  counseling). 

In  order  to  adequately  summarize  psycho- 
therapy research  and  practice,  it  is  necessary  to 
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reflect  components  of  a  number  of  definitions  of 
psychotherapy.  In  this  report,  the  definition  of 
psychotherapy  is  limited  only  in  the  sense  that 
an  effort  is  made  to  view  psychotherapy  as  a 
field  that  is  both  scientific  and  professional. 
Thus,  our  definition  will  emphasize  therapies 
that  have  an  established  scientific  base  and  that 
are  delivered  by  trained  professionals.  In  addi- 
tion, although  it  seems  unnecessary  (as  well  as 
difficult)  to  differentiate  between  levels  of 
psychopathology,  this  report  is  most  concerned 
with  those  disorders  that  cause  severe  dysfunc- 
tion. Although  psychotherapy  may  be  useful  in 


aiding  individuals'  adjustment  and  improving 
the  quality  of  life,  the  primary  concern  here  is 
with  problems  that  require  some  form  of  outside 
intervention. 

The  following  section  briefly  describes  the 
scope  of  psychotherapy  practice,  in  terms  of  the 
definitional  variables  described  above.  It  is 
hoped  that  this  review  will  orient  readers  to  the 
central  features  of  psychotherapy  and  will  iden- 
tify the  central  factors  that  must  be  accounted 
for  in  an  assessment  of  psychotherapy. 


THEORETICAL  ASSUMPTIONS 

When  discussing  psychotherapy,  one  typical- 
ly refers  to  differences  between  treatments  based 
on  different  underlying  theoretical  assumptions. 
There  is  a  wide  variety  of  beliefs  about  people 
and  pathology,  and  this  has  resulted  in  the  de- 
velopment of  a  number  of  different  psychother- 
apeutic treatments.  The  variety  of  psychothera- 
peutic treatments,  at  least  in  terms  of  the  num- 
ber of  labels  used  to  describe  psychotherapies, 
is  somewhat  bewildering.  Nevertheless,  many 
treatments  have  similar  origins,  and  it  is  possi- 
ble to  examine  the  range  of  therapies  by  consid- 
ering only  a  few  major  theoretical  perspectives. 
For  present  purposes,  therapies  based  on  psy- 
chodynamic,  humanistic/ phenomenological, 
and  behavioral  theories  are  discussed  as  the  per- 
spectives that  underlie  most  current  practice. 
These  theoretical  perspectives  do  not  exist  in 
pure  form,  but  they  illustrate  the  most  impor- 
tant differences  between  treatment  approaches 
(see  259). 

Psychodynamic. — The  psychodynamic  per- 
spective, which  grew  out  of  the  classical  psy- 
choanalytic theory  of  Freud  (88),  is  the  oldest 
and  the  most  complex  of  the  three  orientations. 
Although  psychoanalytic  theory  cannot  be  de- 
scribed here  in  detail,  the  role  of  unconscious 
motivation,  one  aspect  of  Freud's  theory,  can 
provide  a  basis  for  understanding  the  nature  of 
psychodynamic  (also  called  analytic)  treat- 
ments. The  importance  of  unconscious  motiva- 
tion is  central  to  all  of  Freud's  psychological  in- 


terpretations. The  driving  forces  of  an  individ- 
ual's life  are  seen  as  originating  in  some  inner 
system  of  which  he  or  she  might,  at  the  most,  be 
only  vaguely  aware.  Most  of  the  unconscious 
cannot  be  called  into  conscious  awareness;  any 
move  in  this  direction  meets  with  resistance,  or 
censorship. 

According  to  Freud's  theory,  some  uncon- 
scious impulses  come  into  conflict  with  envi- 
ronmental constraints  and  moral  prohibitions. 
To  keep  such  threatening  impulses  from  coming 
into  consciousness,  part  of  the  normal  energy  of 
mental  life  is  used  to  provide  a  constant  defense, 
or  resistance,  against  the  acceptance  of  such  im- 
pulses. These  impulses,  however,  are  not  elimi- 
nated by  such  a  defensive  reaction;  instead,  they 
express  themselves  in  indirect  ways,  leading  to 
behavior  that  the  individual  is  sometimes  un- 
able to  explain.  Neurosis  occurs  when  such  be- 
havior is  especially  prominent  in  the  life  of  an 
individual. 

Thus,  the  goal  of  analytic  psychotherapy  is 
gradually  to  promote  the  patient's  insight  into 
his  or  her  underlying  motivations  and  conflicts 
by  repeatedly  working  through  the  patient's  in- 
tellectual and  emotional  defensive  resistances. 
Acquisition  of  insight,  along  with  affective  un- 
derstanding, is  presumed  to  be  a  sufficient  and 
necessary  condition  for  restructuring  significant 
aspects  of  the  patient's  personality  and  for  pro- 
moting the  development  of  more  adaptive  be- 
haviors in  real  life. 
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The  methods  that  use  therapies  based  on  the 
psychodynamic  approach  are  intended  to  aid 
the  patient  in  bringing  unconscious  material  to 
awareness.  The  core  of  therapy  lies  in  trans- 
ference. Freud  suggested  that  patients  inevitably 
project  into  the  therapeutic  situation  feelings 
and  attitudes  from  their  past,  and  that  they 
sometimes  reenact  with  the  therapist  important 
situations  and  traumatic  experiences  repressed 
since  childhood.  By  becoming  aware  of  these 
unresolved  conflicts,  patients  can  master  them 
and  liberate  their  energy  to  develop  a  mature 
personality.  Therapeutic  techniques  are  de- 
signed so  as  to  encourage  and  to  analyze  this 
process  of  transference. 

In  Freudian  psychotherapy,  the  therapist  also 
makes  timely  interpretations  of  the  patient's  free 
associations  and  dreams  and  deals  with  the  pa- 
tient's resistances  to  the  content  of  this  un- 
conscious material.  Psychoanalytic  therapy  has 
provided  the  model  for  most  "verbal"  therapies 
(i.e.,  therapies  based  on  conversations  between 
therapist  and  patient).  Although  infrequently 
practiced  in  pure  form,  in  part  because  many  of 
Freud's  students  (e.g.,  Adler,  Jung)  developed 
their  own  forms  of  therapy,  psychoanalytic  the- 
ory has  had  a  major  influence  on  other  types  of 
psychotherapeutic  treatments. 

Humanistic/Phenomenological. — The  second 
major  orientation  that  has  influenced  present 
day  treatment  can  be  called  the  humanistic/ 
phenomenological  perspective.  The  approach  is 
humanistic  because  it  posits  that  the  basic  aim 
of  the  individual  is  the  achievement  of  personal 
growth.  Personal  growth  is  variously  conceived 
of  as  self-actualization  (e.g.,  234,235,236), 
meaning  in  life  (e.g.,  87),  and  cognitive  com- 
plexity (e.g.,  137).  The  perspective  is  also  called 
phenomenological  because  it  holds  that  the 
central  focus  of  treatment  should  be  an  individ- 
ual's "phenomenal  field."  Thus,  a  person's 
perceptions  about  himself  or  herself  and  the 
world  (rather  than  unconscious  thoughts)  are 
believed  to  be  central  to  understanding.  The 
theory  further  posits  that  each  person  has  a  fun- 
damental urge  to  preserve  his  or  her  self-concept 
(i.e.,  perceptions  about  self). 

This  theoretical  approach,  like  psychoanalyt- 
ic theory,  has  influenced  how  psychotherapy  is 


practiced  by  a  variety  of  different  therapists.  A 
major  form  of  therapy  based  on  this  perspective 
is  "client-centered"  or  "nondirective"  psycho- 
therapy. This  therapy  is  grounded  in  the  early 
theoretical  work  of  Rogers  (234;  see  also  235, 
236).  According  to  Rogers,  when  individuals 
have  needs,  impulses,  or  experiences  that  are 
not  consistent  with  their  self-concept,  tension 
and  anxiety  result.  More  severe  problems  result 
when  individuals  react  with  defensive  mecha- 
nisms to  avoid  awareness  of  their  contradictory 
responses  by  ignoring  important  responses  or 
by  distorting  their  thinking  about  them.  The 
role  of  the  therapist,  according  to  Rogers,  is  to 
be  an  empathetic  responder. 

A  central  tenet  of  client-centered  therapy  is 
the  belief  that  each  individual  has  the  resources 
for  growth.  These  resources  merely  need  to  be 
released  for  the  person  to  resolve  a  problem  and 
achieve  maturity.  Thus,  therapy  facilitates  the 
patient's  growth  by  promoting  the  free  expres- 
sion of  feelings  and  by  refraining  from  imposing 
patterns  and  values.  Out  of  the  therapeutic  rela- 
tionship, patients  (almost  always  referred  to  as 
clients)  evolve  a  new  conceptualization  of  the 
self.  This  view  of  themselves  is  more  tolerant, 
especially  of  their  failings,  and  it  resolves  the 
differences  between  their  ideal  image  and  their 
actual  perception  of  themselves.  As  a  result, 
anxiety  and  dysfunctional  tendencies  are  les- 
sened, and  there  is  greater  objectivity  in  the 
handling  of  reality.  The  new  self-image  should 
also  be  accompanied  by  a  more  harmonious  ex- 
pression of  attitudes  and  feelings. 

Therapists'  efforts  to  interpret,  evaluate,  or 
guide  the  individual  (which  are  central  to  ana- 
lytic therapy)  are  felt  by  humanistic  therapists 
to  hamper  patients'  emerging  sense  of  self- 
growth  and  self-direction.  Techniques  common- 
ly employed  in  client-centered  therapy  are  in- 
tended to  facilitate  individuals  who  seek  help  in 
achieving  their  own  insight.  These  techniques 
include  attentive  listening  to  clients'  commu- 
nications for  content  and  feeling,  reflecting  or 
verbally  focusing  clients'  feelings,  and  encour- 
aging clients  in  their  efforts  to  manage  their  own 
problems. 

Behavioral. — The  third  major  orientation  is 
the  behavioral  perspective.  It  developed,  at  least 
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initially,  from  an  extensive  body  of  experimen- 
tal research  and  theory  associated  with  animal 
and  human  learning  research.  Theorists  of  be- 
havior such  as  Skinner  (263,264,265),  Pavlov, 
and  Hull — who  were  not  psychotherapists  or  di- 
rectly interested  in  the  treatment  of  mental  dys- 
function— are  most  responsible  for  the  ideas 
that  underlie  this  approach.  A  wide  variety  of 
therapeutic  approaches  derived  from  these  basic 
research  efforts  have  been  developed  by  later 
theorists  such  as  Wolpe  (302)  and  Bandura  (11). 

The  basic  assumption  of  behavioral  theories 
is  that  most  persistent  pathological  behavior  is 
acquired  through  a  process  of  learning.  In  es- 
sence, experiences  in  the  individual's  life  in- 
stigate patterns  of  behavior  which  remain  unless 
other  conditioning  experiences  modify  them. 
Therapies  based  on  behavior  theory  usually  be- 
gin with  systematic  analysis  of  the  problematic 
situation  and  an  identification  of  the  specific 
behavioral  or  situational  factors  to  be  modified. 
Once  it  is  known  how  and  under  what  condi- 
tions these  behaviors  are  maintained  in  the  indi- 
vidual's behavioral  repertoire,  the  therapist  in- 
terferes directly  with  this  preestablished  order. 
Internal  (cognitive)  or  external  (environmental) 
patterns  of  behavior  contrary  to  the  well-being 
of  the  patient  are  modified  by  systematic  re- 
training procedures. 

Current  clinical  practice  in  behavioral  ther- 
apy is  derived  from  two  laboratory-developed 
theories  of  learning.  The  first,  operant  condi- 
tioning, is  derived  directly  from  research  by 
Skinner  (e.g.,  263).  Because  behavior  is  pre- 
sumed to  be  lawful  and  predictable,  behavior- 
ists  believe  it  is  controllable  once  the  relevant 
reinforcing  variables  (those  that  maintain  the 
behavior)  have  been  determined.  Operant  con- 
ditioning includes  aversive  counterconditioning 
to  eliminate  undesirable  behaviors  and/or  posi- 
tive reinforcement  to  establish  and  maintain 
new  adaptive  behaviors.  Aversive  countercon- 
ditioning entails  pairing  the  patient's  behavioral 
deviation  with  an  unpleasant  stimulus.  For  ex- 
ample, alcoholism  has  been  treated  by  adminis- 
tering a  substance  to  patients  which  would  in- 
duce violent  and  uncomfortable  physical  reac- 
tions in  the  presence  of  alcohol.  Positive  rein- 
forcement entails  rewarding  the  patient  for  pro- 


ducing desired  behaviors.  For  instance,  positive 
reinforcement  has  been  used  to  treat  autistic 
children  who  are  disassociated  from  reality  and 
unable  to  engage  in  gratifying  interpersonal  re- 
lationships. These  children  are  rewarded  with 
verbal  approval  and  candy  when  they  show  any 
evidence  of  social  responsiveness  or  interaction. 

Another  form  of  clinical  practice,  developed 
by  Wolpe  (302),  is  reciprocal  inhibition.  This 
therapy  is  an  outgrowth  of  Pavlov's  classical 
conditioning  studies  and  of  Hull's  learning 
theory.  The  theory  underlying  reciprocal  inhibi- 
tion is  that  neuroses  (in  particular,  phobias)  are 
characterized  by  persistent  anxiety  responses  to 
situations  in  which  there  is  no  objective  danger. 
The  neuroses  are  produced  when  an  individual 
associates  high  intensities  of  anxiety  and  neutral 
stimulus  events.  Therapy  based  on  principles  of 
reciprocal  inhibition  systematically  desensitizes 
the  patient  to  particular  stimulus  events.  Thus, 
a  patient  with  neurotic  anxiety  is  taught  to  re- 
lax. Then,  in  a  state  of  relaxation,  he  or  she  is 
asked  to  think  about  a  mildly  anxiety-provok- 
ing situation.  This  technique  is  then  used  with 
increasingly  more  disturbing  situations.  Relaxa- 
tion inhibits  anxiety,  and  the  patient  is  progres- 
sively able  to  master  the  situations  previously 
associated  with  anxiety  reactions. 

Although  many  of  the  behavioral  therapies 
rely  directly  on  learning  paradigms  tested  with 
laboratory  animals,  a  more  recent  trend  (in 
part,  exemplified  by  Bandura's  work  (11,12))  is 
the  development  of  cognitive  behavior  thera- 
pies. Within  such  therapies,  laws  of  learning  are 
used  as  the  basis  for  modifying  cognitive  proc- 
esses (see  164,  180).  The  patient  in  cognitive  be- 
havior therapy  takes  an  active  role  in  identify- 
ing problems  and  working  with  the  therapist  to 
develop  mechanisms  to  control  desirable  or  un- 
desirable thoughts.  Bandura's  (12)  social  learn- 
ing theory  is  a  related  approach  that  emphasizes 
the  role  of  observing  others  (models)  who  are 
reinforced  or  punished.  There  are  also  a  variety 
of  eclectic  behavior  therapies  (e.g.,  154)  which 
employ  a  varying  set  of  learning  principles. 

Common  Factors. — Psychotherapy,  as 
should  be  clear  from  the  above,  is  derived  from 
a  range  of  assumptions.  Although  the  three 
theoretical  perspectives  described  above  under- 
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lie  most  forms  of  therapy,  there  is  considerable 
eclecticism  in  practice.  A  practitioner  is  likely  to 
hold  some  views  or  use  techniques  that  are  con- 
sistent with  one  dominant  theoretical  perspec- 
tive and  to  hold  other  views  and  use  other  tech- 
niques that  are  quite  inconsistent  with  it. 

In  part,  the  reason  for  this  eclecticism  is  that  a 
number  of  important  similarities  exist  across 
different  theoretical  persuasions.  Some  theorists 
such  as  Frank  (84,86),  in  fact,  argue  that  psy- 
chotherapeutic change  is  predominantly  a  func- 
tion of  factors  common  to  all  therapeutic  ap- 
proaches (e.g.,  84).  The  primary  ingredients  of 
such  common,  nonspecific,  factors  are  the  ther- 


apist's understanding,  respect,  interest,  encour- 
agement, and  acceptance.  Thus,  while  the  con- 
tents and  procedures  of  psychotherapy  may  dif- 
fer across  theoretical  orientations,  all  forms  of 
psychotherapy  share  common  "healing"  func- 
tions. All  therapists  combat  the  patient's  demor- 
alization and  sense  of  hopelessness  by  the  rela- 
tionship they  establish  with  the  patient  and  by 
providing  an  explanation  for  previously  inex- 
plicable feelings  and  behavior.  According  to 
those  who  maintain  that  such  nonspecific  fac- 
tors are  responsible  for  psychotherapy's  effects, 
one  reason  for  the  success  of  therapy  is  because 
it  removes  the  mystery  from  the  patient's  suffer- 
ing and  supplants  it  with  hope. 


MENTAL  DISORDERS 

Although  it  is  true  that  one  source  of  confu- 
sion about  psychotherapies  is  the  divergent 
theoretical  assumptions  of  therapists,  it  should 
also  be  recognized  that  the  nature  of  the  prob- 
lems dealt  with  by  specific  therapies  varies 
widely.  Not  only  are  a  broad  range  of  problems 
considered  psychopathological,  but  for  any  one 
individual,  the  identification  of  a  disorder  is 
often  dependent  on  his  or  her  own  reports  of 
symptoms.  In  contrast  to  physical  diseases, 
which  are  usually  accompanied  by  measurable 
physiological  changes,  mental  disorders  must 
usually  be  identified  by  a  patient's  or  another 
person's  report  of  a  problem.  Notwithstanding 
the  many  exceptions  to  this  distinction  between 
physical  and  mental  illness,  the  difficulties  of  di- 
agnosis of  mental  illness  are  widely  recognized. 
An  additional  problem  in  diagnosis  is  that  the 
severity  of  a  mental/psychological  problem  is 
based  on  the  context  in  which  it  occurs.  A  prob- 
lem that  may  be  diagnosed  as  a  severe  disorder 
in  one  situation  (e.g.,  at  work)  may  be  consid- 
ered a  minor  disturbance  in  another  context 
(e.g.,  at  home). 

Widely  varying  phenomena  have  been  con- 
sidered mental  disorders.  Most  therapists  and 
researchers  seem  to  contend  that  any  problem 
which  causes  a  patient  discomfort  and  dysfunc- 
tion is  a  mental  health  problem  requiring  treat- 
ment. For  present  purposes,  the  most  important 
aspect  of  the  problem  would  seem  to  be  the  level 


of  dysfunction.  Any  person  who  is  unable  to 
carry  out  his  or  her  normal  responsibilities  is  a 
potential  candidate  for  therapy.  Level  of  dys- 
function, however,  is  a  difficult  distinction  to 
make.  An  individual  may  appear  to  be  function- 
ing well,  but  may  be  extremely  troubled.  One 
would  want  to  alleviate  such  dysfunction  and 
prevent  more  serious  dysfunction. 

Below,  the  disorders  treated  by  psychothera- 
py have  been  grouped  within  three  general  cate- 
gories: neuroses,  psychoses,  and  conduct  disor- 
ders. While  the  most  current  diagnostic  manual 
(63)  lists  mental  disorders  somewhat  differently, 
for  the  purpose  of  describing  the  range  of  men- 
tal disorders,  this  organization  seems  more 
appropriate. 

Psychoses  are  severe  disorders  characterized 
by  grossly  illogical  thought  patterns  and  dis- 
torted perceptions  of  reality.  The  psychotic  in- 
dividual may  have  prolonged  periods  of  melan- 
choly or  elation  and  may  exhibit  extreme  isola- 
tion or  withdrawn  behavior.  The  symptoms  of  a 
schizophrenic  disorder,  the  most  prevalent  type 
of  psychosis,  include  bizarre  delusions,  auditory 
hallucinations,  and  incoherence.  Schizophrenia 
is  accompanied  by  a  deterioration  in  level  of 
functioning  in  areas  such  as  work,  social  rela- 
tions, and  self-care. 

Neuroses,  the  most  common  disorders,  are 
problems  marked  by  feelings  of  anxiety,  fear, 
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depression,  guilt,  and  other  unpleasant  emo- 
tions. Anxiety,  for  example,  is  related  to  phys- 
ical symptoms  such  as  trembling,  sweating,  and 
heart  pounding.  It  may  result  in  the  individual's 
feeling  continually  worried  and  being  overly  at- 
tentive. The  individual  may  feel  impatient  and 
irritable,  and  may  have  insomnia.  While 
associated  with  discomfort,  such  anxiety  may 
not  impair  social  or  occupational  functioning 
more  than  mildly.  In  contrast,  agoraphobia  (an 
irrational  fear  of  crossing  or  being  in  open 
spaces)  causes  individuals  to  avoid  certain  situa- 
tions, and  thus  may  interfere  seriously  with 
social  and  occupational  functioning.  Neurotic 
conflicts  may  also  be  the  basis  of  other  prob- 
lems, such  as  divorce  and  child  abuse. 

Conduct  disorders  are  characterized  by  a  lack 
of  normal  feelings  and  conscience  that  results  in 
behaviors  which  violate  basic  social  rules.  Con- 
duct disorders  are  reflected  by  problems  such  as 
uncontrolled  impulses  (e.g.,  outbursts  of  aggres- 
siveness, failure  to  resist  impulses  to  steal  or  to 
set  fires)  and  drug  and  alcohol  abuse.  Conduct 
disorders  among  children  and  adolescents  in- 
clude such  symptoms  as  physical  violence 
against  persons  or  property  (e.g.,  vandalism, 
rape,  mugging),  chronic  violations  of  rules 
(e.g.,  truancy  from  school,  alcohol  or  drug 
abuse),  and  persistent  lying  and  stealing. 

In  addition  to  the  above  disorders,  there  are  a 
variety  of  conditions  which  reflect  interactions 
between  different  types  of  problems.  Most 
prominent  here  are  psychosomatic  disorders, 
where  physical  symptoms  are  thought  to  have  a 
psychological  base  (e.g.,  188).  Some  have  sug- 
gested (e.g.,  152)  that  over  50  percent  of  the  pa- 
tients seen  in  general  medical  practice  have  psy- 


chologically based  problems.  Conditions  such 
as  asthma,  colitis,  anorexia  nervosa,  as  well  as 
physical  problems  such  as  obesity  and  head- 
aches due  to  stress,  have  been  considered  suit- 
able problems  for  psychotherapeutic  treatment. 

The  severity  of  any  psychopathological  disor- 
der depends,  in  part,  on  the  individual's  reac- 
tion. For  example,  whereas  depression  in  some 
individuals  may  be  associated  with  poor  appe- 
tite, insomnia,  and  loss  of  energy,  in  others  it 
may  be  associated  with  an  attempted  suicide. 
Individuals  also  differ  in  the  degree  of  subjective 
distress  they  experience  and  the  urgency  with 
which  they  desire  and  need  help.  Whereas  for 
some  individuals,  mild  depression  is  so  bother- 
some that  they  cannot  function,  others  who 
appear  very  depressed  carry  out  normal  lives. 
Whereas  some  individuals  who  seem  only  mild- 
ly troubled  feel  in  great  need  of  therapeutic  help, 
others  who  have  suffered  major  trauma  are  re- 
sistant to  seeking  help. 

The  diagnosis  of  a  disorder,  as  well  as  its 
seriousness,  is  thus  inevitably  part  of  a  subjec- 
tive decisionmaking  process  on  the  part  of  the 
therapist.  The  therapist  is  usually  responsible 
for  the  diagnostic  decision  and  is  influenced 
both  by  the  patient's  report  of  the  problem  and 
by  objective  evidence  (e.g.,  data  on  the  patient's 
inability  to  communicate  with  others).  As  de- 
scribed in  the  next  section,  particular  therapists 
may  view  these  problems  differently,  some  fo- 
cusing on  behavioral  disorders,  others  on  dys- 
functional thoughts  and  emotions.  These  differ- 
ent emphases  sometimes  make  it  difficult  to 
compare  effectiveness  across  psychotherapies 
and  psychotherapists. 


PRACTITIONERS 

As  indicated  above,  the  delivery  of  psycho- 
therapy differs  not  only  in  terms  of  theoretical 
assumptions  and  types  of  disorders  treated,  but 
also  in  terms  of  the  personnel  who  offer  ther- 
apy. Therapists  have  an  important  influence  on 
the  nature  of  psychotherapeutic  treatment.  Dif- 
ferences in  therapists'  theoretical  orientation,  or 
preferred  technique,  as  well  as  differences  in 


their  training,  contribute  to  the  difficulty  of  de- 
scribing and  assessing  psychotherapy. 

Professional  therapists  differ  in  a  number  of 
ways,  both  related  and  unrelated  to  their  train- 
ing and  disciplinary  orientation.  Thus,  thera- 
pists may  differ  with  regard  to  their  underlying 
assumptions  concerning  psychotherapy.  They 
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may  adopt  a  psychodynamic,  humanistic/phe- 
nomenological,  or  behavioral  perspective,  or  an 
eclectic  set  of  beliefs  based  on  several  major 
orientations.  Therapists  also  differ  in  the  mo- 
dalities and  techniques  they  employ  to  provide 
treatment.  For  example,  practitioners  may  per- 
form therapy  mainly  with  individuals,  groups, 
or  families;  as  well,  they  may  focus  more  on 
early  events  in  the  patient's  life  or  on  recent 
events,  or  lead  a  patient  to  insights  concerning 
behavior  or  feelings.  These  differences  in  ap- 
proach are  not  clearly  associated  with  the  thera- 
pist's professional  training  and  identification. 

The  nature  of  the  relationship  that  a  therapist 
establishes  with  a  patient  may  be  a  particularly 
important  factor  in  the  nature  of  changes  caused 
by  psychotherapy.  The  patient  reacts  to  the 
therapist  in  a  general  way,  not  necessarily  to 
specific  characteristics.  Thus,  therapists  and  re- 
searchers have  been  very  concerned  with  the 
therapist  as  a  human  being  (see  274).  Profes- 
sional therapists  obviously  differ  in  terms  of 
their  personal  characteristics.  Some  of  these 
characteristics,  such  as  sensitivity,  empathy,  ac- 
tiveness,  and  involvement,  are  seen  by  some 
theorists  (e.g.,  86)  as  essential  to  successful 
therapeutic  outcomes.  Therapists  also  differ 
with  regard  to  age,  sex,  cultural  background, 
ethnic  factors,  level  of  professional  experience, 
psychological  sophistication,  social  and  cultural 
values,  and  cognitive  styles.  Any  of  these  char- 
acteristics may  have  a  significant  effect  on  the 
way  in  which  psychotherapy  is  delivered.  These 
characteristics  may  interact  with  the  practi- 
tioner's theoretical  perspective  to  produce  a 
unique  therapeutic  treatment. 

For  the  purposes  of  this  report,  it  seems  im- 
portant to  distinguish  professional  from  non- 
professional helpers.  While  it  is  sometimes  dif- 
ficult to  make  distinctions  between  what  profes- 
sionals and  nonprofessionals  actually  do  in 
treatment  settings,  the  professional  usually  has 
a  very  different  orientation  and  different  re- 
sponsibilities from  those  of  the  nonprofessional. 
Obviously,  persons  with  mental  distress  can  be 
helped  by  a  variety  of  other  individuals,  includ- 
ing family  and  friends.  These  other  individuals 
may  assist  the  patient  in  understanding  his  or 
her  problems  and  in  making  behavioral  adjust- 


ments. The  unique  contribution  of  profession- 
ally trained  therapists  is  important  to  specify. 

The  primary  professionals  who  deliver  psy- 
chotherapy are  psychiatrists,  clinical  psycholo- 
gists, and  psychiatric  social  workers  (see  118). 
In  1977,  there  were  approximately  28,000  psy- 
chiatrists professionally  active  in  the  United 
States.  Of  this  number,  approximately  24,500 
spend  at  least  half  of  their  time  in  patient  care; 
the  others  are  primarily  administrators,  teach- 
ers, or  researchers  (5).  Psychiatrists  are  medical 
school  graduates  who  have  had  a  1-year  intern- 
ship and  a  2-  to  3-year  residency  in  psychiatry 
at  an  approved  hospital  or  clinic.  In  addition, 
they  must  have  practiced  their  specialty  for  2 
years  and  be  tested  in  order  to  qualify  for  cer- 
tification by  the  American  Board  of  Psychiatry 
and  Neurology. 

One  problematic  aspect  (103)  to  understand- 
ing the  delivery  of  psychotherapy  is  that  many 
nonpsychiatric  physicians  also  provide  psycho- 
therapy services.  Thus,  according  to  the  Na- 
tional Institute  of  Mental  Health  (see  225),  over 
half  of  the  patients  seen  for  psychiatric  prob- 
lems are  treated  by  primary  care  physicians.  It  is 
important  to  emphasize,  however,  that  while 
many  nonpsychiatric  physicians  are  involved  in 
mental  illness  diagnosis  and  treatment,  their  in- 
volvement may  be  very  limited  (see  60). 

In  terms  of  psychologists,  according  to  a  1976 
membership  survey  by  the  American  Psycho- 
logical Association,  there  were  approximately 
23,000  psychologists  engaged  in  mental  health 
activities.  These  psychologists  worked  in  indi- 
vidual or  group  practice  settings,  mental  health 
centers,  general  or  psychiatric  hospitals,  and 
outpatient  clinics.  A  1977  survey  (183)  found 
similar  results,  although  it  uncovered  a  slightly 
higher  number  (approximately  26,000)  of  clin- 
ical psychologists  (because  nonmembers  of  the 
Association  were  located). 

Generally,  clinical  psychologists  hold  a  doc- 
toral degree  (either  a  Ph.  D.  or  a  Psy.  D.), 
although  some  (less  than  10  percent)  hold  only  a 
master's  degree.  An  integral  part  of  the  training 
of  clinical  psychologists  is  a  1-year  internship  in 
a  clinic  or  hospital  setting  approved  by  the 
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American  Psychological  Association;  in  addi- 
tion, clinical  psychologists  typically  have 
several  years  supervised  experience,  and  Ph.  D.s 
receive  training  in  research  (which  culminates  in 
a  dissertation).  A  psychologist  who  has  had  at 
least  5  years  cf  postdoctoral  experience  may  ap- 
ply for  advanced  certification  from  the  Amer- 
ican Board  of  Professional  Psychology. 

A  large  number  of  psychotherapists  are  clini- 
cal social  workers.  In  1978,  10,922  social  work- 
ers were  included  in  the  Register  of  Clinical 
Social  Workers  (see  2);  in  addition,  many  other 
social  workers  who  are  members  of  the  National 
Association  of  Social  Workers  practice  psycho- 
therapy. Psychiatric  social  workers  have  had  2 
to  3  years  of  postgraduate  training  in  a  school  of 
social  work  which  typically  includes  an  intern- 
ship. Psychiatric  social  workers  with  a  masters 
degree  in  social  work  and  2  years  of  experience 
may  take  a  test  for  certification  by  the  Academy 
for  Certification  of  Social  Workers. 

Although  the  largest  number  of  identifiable 
psychotherapists  are  either  psychiatrists,  psy- 
chologists, or  psychiatric  social  workers,  vari- 
ous other  professionals  are  trained  in  psycho- 
therapy and  administer  it  to  patients.  These 
other  professionals  include  psychiatric  nurses, 


DELIVERY  SETTINGS 

Variations  in  psychotherapy  are  also  seen  in 
the  diversity  of  settings  in  which  therapy  takes 
place.  These  settings  may  have  an  important  im- 
pact on  the  nature  of  psychotherapy.  In  some 
cases,  the  setting  influences  the  type  of  therapy 
available  to  the  patient.  Settings  may  include 
traditional  inpatient  mental  hospitals,  commu- 
nity mental  health  centers  (CMHCs),  private  of- 
fices, residential  treatment  centers,  and  other 
locations  such  as  schools,  offices,  and  military 
facilities. 

Prior  to  the  establishment  of  CMHCs  in  1963, 
most  psychotherapy  was  provided  in  mental 
hospitals;  certainly,  these  settings  provided 
much  of  the  societally  supported  mental  health 
treatment.  Although  there  is  some  controversy 


pastoral  counselors,  educational  psychologists, 
and  occupational  therapists,  as  well  as  a  number 
of  individuals  from  other  disciplines.  Nurses,  in 
particular,  play  an  important  role  in  the  provi- 
sion of  psychotherapy.  According  to  a  1972  sur- 
vey (see  2),  almost  38,000  registered  nurses 
reported  working  in  a  "psychiatric/mental 
health  area;"  about  4,000  of  them  had  master's 
degrees.  These  nurses  and  other  professionals 
have  had  unique  disciplinary  training,  but  they 
have  also  completed  internships  with  (or  under 
the  supervision  of)  psychiatrists  and  psycholo- 
gists, and  they  often  provide  treatment  in  com- 
bination with  other  professional  therapists.  To 
assess  the  effects  of  psychotherapy,  the  role  of 
these  professionals  needs  to  be  recognized. 

In  addition,  the  work  of  paraprofessionals 
(i.e.,  psychiatric  aides  and  trained  volunteers) 
should  be  noted.  The  Alcohol,  Drug  Abuse,  and 
Mental  Health  Administration  (2)  has  estimated 
that  over  150,000  paraprofessionals  work  in 
direct  contact  with  mental  patients.  Many  psy- 
chotherapy settings  rely  on  the  services  of  per- 
sonnel who,  though  not  formally  trained  as 
therapists,  have  a  great  deal  of  contact  with  pa- 
tients. Such  paraprofessionals  may  play  a  role 
in  identifying  patient  problems  and  maintaining 
therapy  begun  by  the  professional  therapist 
(4,65). 


as  to  the  relationship  between  the  development 
of  CMHCs  and  deinstitutionalization  of  mental 
hospital  patients,  there  has  been  a  sharp  decline 
in  the  number  of  hospital  beds  for  mental  pa- 
tients. In  1962,  psychiatric  hospitals  had  a  total 
of  717,000  beds  with  91.0-percent  occupancy 
(194);  in  1978,  there  were  only  219,517  beds 
with  80.7-percent  occupancy  (121).  Although 
large  numbers  of  the  severely  disturbed  are  still 
hospitalized,  their  stays  are  generally  much 
shorter  than  was  previously  the  case.  The  na- 
ture of  treatment  has  also  changed.  Patients  to- 
day have  more  frequent  access  to  specific  treat- 
ments (i.e.,  treatment  other  than  being  "housed" 
in  the  hospital),  including  group  therapy,  indi- 
vidual therapy,  and  psychoactive  drugs  (i.e., 
substances  that  alter  mental  states).  In  addition, 
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patients  are  usually  exposed  to  a  number  of  dif- 
ferent therapists  (e.g.,  a  psychiatrist  who  super- 
vises their  use  of  drugs,  a  psychologist  who  does 
individual  counseling,  and  a  psychiatric  nurse 
or  social  worker  who  leads  group  therapy). 

CMHCs  and  other  outpatient  services  repre- 
sent the  fastest  growing  segment  of  the  mental 
health  delivery  system.  Since  their  inception  in 
1963,  CMHCs  have  been  made  available  to  al- 
most 50  percent  of  the  U.S.  population.  In  1975, 
according  to  the  National  Institute  of  Mental 
Health  (197),  patient  visits  to  CMHCs  totaled 
1,961,000.  As  do  outpatient  mental  hospitals, 
CHMCs  use  a  variety  of  mental  health  profes- 
sionals, and  a  patient  is  likely  to  have  exposure 
to  several  modes  of  treatment.  CMHCs  tend  to 
emphasize  short-term  treatment,  even  though 
they  often  deal  with  chronic  patients. 

A  large  segment  of  mental  health  treatment 
takes  place  in  private  offices.  Once  available 
only  to  the  well-to-do,  private  practice  has 
flourished  recently  as  public  and  private  in- 
surance coverage  for  mental  health  disorders 


has  become  more  widespread.  It  is  estimated 
that  there  are  now  15,562  physicians,  4,700 
psychologists,  and  2,189  social  workers  engaged 
more  than  half-time  in  the  private  practice  of 
psychotherapy  (2). 

Several  other  settings  for  psychotherapy  are 
worth  noting.  Outpatient  mental  health  services 
are  growing  in  acceptance  as  part  of  medical 
primary  care.  Health  maintenance  organiza- 
tions, hospital-based  outpatient  clinics,  and 
even  some  small  physician  groups  are  including 
mental  health  professionals  on  their  staffs  (e.g., 
26).  In  general,  such  services  are  used  to  supple- 
ment medical  treatments  and/or  to  reduce  reli- 
ance on  physical  health  care.  Much  psychother- 
apy also  goes  on  in  nonpsychiatric  residential 
treatment  centers.  For  example,  under  some  cir- 
cumstances, homes  for  children  with  behavioral 
and  learning  problems  and  prisons  can  be  con- 
sidered treatment  centers.  In  addition,  many  or- 
ganizations such  as  schools,  industries,  and  the 
military  provide  in-house  resources  to  deliver 
psychotherapy  for  members  with  mental  health 
problems. 


SCOPE  OF  THIS  REPORT 

As  should  be  clear  at  this  point,  psychothera- 
py refers  to  a  broad  range  of  treatments  used  to 
ameliorate  a  number  of  different  kinds  of  condi- 
tions, by  different  therapists,  and  in  a  variety  of 
settings.  The  present  report,  whose  goal  is  to  de- 
scribe the  scientific  basis  for  assessing  psycho- 
therapy, attempts  to  delimit  its  discussion  in 
two  ways. 

First,  the  report  attempts  to  describe  those 
aspects  of  psychotherapy  that  have  the  most  im- 
portant implications  for  financing  and  reim- 


bursement policy.  This  is  reflected  in  an  empha- 
sis on  severe  dysfunctions  (which  are,  potential- 
ly, the  most  costly)  and  an  effort  to  distinguish 
the  applicability  of  psychotherapy  to  particular 
problems.  Second,  the  report  attempts  to  de- 
scribe the  current  range  of  practice  and  research 
on  psychotherapy  by  referring  to  the  four 
categories  of  variables  described  above.  Thus, 
the  theoretical  assumptions  of  therapy,  and  the 
characteristics  of  patients,  therapists,  and 
delivery  settings  serve  as  categories  that  need  to 
be  considered  in  psychotherapy  assessments. 
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Given  the  diversity  of  theoretical  approaches 
to  psychotherapy,  as  well  as  the  range  of  mental 
health  problems,  therapists,  and  delivery  set- 
tings, it  is  difficult  to  give  simple  answers  to 
questions  about  psychotherapeutic  efficacy.  Al- 
though the  position  adopted  here  will  be  that  it 
is  possible  to  answer  efficacy  questions  through 
scientific  research,  such  questions  are  un- 
doubtedly difficult  to  answer.  Both  because  of 
the  inherent  nature  of  psychotherapy  and  be- 
cause of  the  nature  of  the  scientific  research 
process,  answers  to  questions  about  psychother- 
apy will  require  a  complex  sequence  of  steps  and 
the  adaptation  of  several  research  technologies 
to  the  problem  of  psychotherapy. 

Questions  about  psychotherapy  are  complex, 
because,  for  policy  purposes,  they  have  been 
stated  very  globally.  It  must  be  recognized  that 
general  statements  as  to  whether  psychotherapy 
is  effective  are  necessarily  equivocal  and  must 
be  tempered  by  information  as  to  the  specific 
conditions  under  which  particular  treatments 
are  efficacious.  The  way  in  which  the  level  of 
specificity  of  one's  question  affects  research  on 
psychotherapy  and  the  methodological  consid- 
erations that  affect  the  conduct  and  interpreta- 
tion of  efficacy  research  are  described  below. 

Most  research  assessing  the  effectiveness  of 
psychotherapy  has  examined  very  specific  is- 
sues. Which  technique  is  more  effective  and  how 
effectiveness  is  moderated  by  differences  among 
patients,  therapists,  and  settings  are  the  typical 
foci  of  psychotherapy  outcome  research  (see 
207,287).  Much  of  this  research  shows  that 
some  techniques  are  more  effective  than  others, 
although,  unfortunately,  no-treatment  or  place- 
bo treatment  control  conditions  are  not  always 
included  as  part  of  these  studies.  Without  such 
comparison  conditions,  their  implications  for 
the  ultimate  effectiveness  questions  are  difficult 
to  assess. 


Questions  about  the  effectiveness  of  specific 
types  of  psychotherapy  usually  deal  with  the 
conditions  under  which  therapy  is  provided. 
Thus,  the  type  of  mental  dysfunction,  the  char- 
acteristics of  the  patient,  and  the  characteristics 
of  the  delivery  system  are  the  central  variables 
being  tested.  Although  this  research  can  yield 
generalizations  to  policy  about  psychotherapeu- 
tic treatment,  the  inherent  limitations  should  be 
recognized.  Unless  generalizability  has  been  em- 
pirically established  by  tests  conducted  with  a 
range  of  patients  in  an  actual  treatment  site,  the 
conclusions  must  be  regarded  as  tenuous. 

A  number  of  methodological  issues  that  arise 
in  assessing  the  efficacy  of  psychotherapy  are 
described  in  the  following  sections.  The  first 
issue  has  to  do  with  how  one  measures  the  out- 
comes of  therapy.  There  is  a  substantial  litera- 
ture describing  procedures  for  determining  the 
presence  and  strength  of  particular  results  of 
therapy  (see  293).  This  literature  is  examined 
below  in  terms  of  the  development  of  useful  pol- 
icy data  about  psychotherapy.  A  second  set  of 
methodological  issues  concerns  the  design  of 
psychotherapy  research  and  the  confidence  one 
can  have  that  obtained  changes  are  a  result  of  a 
particular  psychotherapeutic  intervention.  The 
research  design  problems  have  to  do  with  deter- 
mining the  reason  for  outcomes  and  organizing 
research  so  that  extraneous  factors  can  be  ruled 
out  as  the  cause  of  treatment  effects  (see  41, 
105).  Also  considered  below  are  the  problems  of 
actually  conducting  psychotherapy  research,  in- 
cluding the  ethical  and  pragmatic  issues  of  em- 
ploying random  assignment  procedures.  A  sepa- 
rate set  of  methodological  problems  having  to 
do  with  the  synthesis  and  interpretation  of  mul- 
tiple efficacy  studies  is  also  described.  A  number 
of  techniques  have  been  developed  for  review- 
ing and  integrating  findings,  and,  potentially, 
these  methods  may  allow  more  definitive  assess- 
ments of  the  psychotherapy  literature. 
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MEASURES  OF  PSYCHOTHERAPEUTIC  OUTCOMES 


Measuring  the  outcomes  of  psychotherapeu- 
tic treatment  has  been  a  major  focus  of  psycho- 
therapy theory  and  research  for  at  least  the  last 
two  decades.  Developing  a  technology  for  meas- 
uring outcomes  is  the  first  step  in  determining 
psychotherapy 's  efficacy.  It  involves  decisions 
about  what  variables  are  important  to  assess,  as 
well  as  the  development  of  measurement  tech- 
niques that  can  be  used  in  actual  treatment  set- 
tings. Because  psychotherapy  takes  a  number  of 
forms  (e.g.,  treatment  based  on  different  theo- 
retical assumptions),  much  of  the  literature 
deals  with  the  selection  of  outcome  measures 
that  are  appropriate  to  the  goals  of  the  treat- 
ment under  study.  One  recent  focus  has  been  the 
development  of  measurement  procedures  that 
can  be  used  across  different  types  of  psycho- 
therapy. Some  of  this  literature  has  proposed 
"batteries"  of  instruments  (e.g.,  293),  whose  in- 
tent is  to  capture  (through  the  use  of  several 
types  of  measurement  procedures)  the  core 
changes  that  result  from  any  application  of 
psychotherapy. 

Underlying  the  effort  to  develop  such  com- 
mon measures  is  a  belief  that  no  one  instrument 
or  set  of  procedures  can  measure  all  the  out- 
comes of  psychotherapy  (21).  Different  psycho- 
therapeutic techniques  applied  by  a  range  of 
therapists  to  various  patient  populations  may 
require  different  measures  of  outcome.  Thus, 
functional  measures  of  behavioral  effects,  al- 
though perhaps  appropriate  to  assess  behavioral 
therapies,  might  be  inappropriate  to  assess  psy- 
chodynamically  based  therapies.  Similarly,  cog- 
nitive measures  might  be  seen  as  inappropriate 
to  evaluate  behavioral  therapies. 

Notwithstanding  the  unique  goals  of  partic- 
ular therapies,  there  seems  to  be  support  for  the 
concept  that  many  of  the  changes  produced  by 
psychotherapy  can  be  assessed  along  some  com- 
mon matrix.  Probably,  this  implies  the  use  of  a 
matrix  that  includes  both  behavioral  and  cogni- 
tive variables.  Any  single  study  of  psychother- 
apy, thus,  would  incorporate  a  number  of  meas- 
ures, not  necessarily  tied  to  the  goals  of  the 
therapy.  The  use  of  such  multiple  common  out- 
comes also  makes  monitoring  potential  detri- 


mental effects  more  possible.  Such  effects  may 
not  be  detectable  if  unique  therapy-relevant 
measures  are  used. 

Measurement  Criteria 

To  be  useful  in  an  effectiveness  analysis, 
measures  of  psychotherapy  outcome  must  be 
both  reliable  and  valid  (see  201).  Reliability 
means  that  the  measure  gives  the  same  finding 
over  multiple  uses  (assuming  no  change  in  what 
is  being  measured)  and  provides  the  same  find- 
ings when  used  by  different  researchers.  Valid- 
ity means  that  the  measure  assesses  the  out- 
comes that  it  is  supposed  to  measure  and  pro- 
vides data  that  are  generalizable  (see,  e.g.,  48). 
Neither  reliability  nor  validity  is  intended  as  a 
theoretical  concept;  each  is  established  by  pre- 
testing the  measuring  instrument. 

The  use  of  reliability  and  validity  criteria 
results  in  particular  measurement  processes'  be- 
ing validated  to  measure  specific  therapeutic  ef- 
fects. Thus,  for  example,  a  single  instrument 
might  not  be  reliable  and  valid  for  assessing  im- 
provement in  both  depression  and  agoraphobia; 
reliability  and  validity,  at  least,  would  have  to 
be  separately  established  for  each  condition. 
Moreover,  outcomes  have  different  meanings  to 
patients,  the  therapist,  and  interested  third  par- 
ties. Validity  may  depend  on  who's  perspective 
one  adopts  in  assessing  the  measurement  instru- 
ment. Below,  some  of  the  differences  in  meas- 
ures designed  to  collect  data  from  various  indi- 
viduals affected  by  therapy  are  described. 

Measures  From  Patients 

Reports  and  ratings  by  patients  of  their  be- 
havior, thoughts,  and  feelings  represent  one 
typical,  and  usually  important  outcome  meas- 
ure. Often,  such  data  are  collected  on  question- 
naires or  through  interviews.  These  measures 
structure  verbal  reports  of  the  patient's  ability 
to  cope  with  various  problems,  and  include 
paper-and-pencil  measures  of  personality  and 
adjustment  (31).  Although  patient  assessment 
measures  yield  important  data,  they  have  ob- 
vious limitations.   These  limitations  include 
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social  desirability  effects  (patients'  responding 
to  create  a  certain  impression)  and  response  bias 
(e.g.,  positive  responding  to  express  apprecia- 
tion to  the  therapist).  Validation  using  other 
types  of  measures  can  determine  the  effects  of 
such  response  patterns. 

One  type  of  patient  measure  is  a  functional 
self-report  instrument  which  asks  patients  to 
report  on  aspects  of  their  daily  functioning 
(123).  Although  narrative  reports  of  functioning 
may  be  collected  as  a  part  of  any  therapeutic 
treatment,  instruments  that  incorporate  stand- 
ardized questions  have  been  used  to  systematize 
this  data  collection.  Such  an  instrument  may  in- 
clude a  series  of  structured  questions  about  time 
lost  from  work,  feelings  of  guilt,  and  satisfac- 
tion with  therapy.  Careful  pretesting  of  these 
questions  (e.g.,  by  comparing  responses  of  indi- 
viduals known  to  be  psychologically  impaired 
with  those  not  impaired)  yields  a  subset  of  these 
questions.  Responses  can  be  quantified  and 
summed  to  form  composite  scores  (total  and 
subset)  of  social  adjustment  and  functioning.  A 
number  of  these  instruments  have  been  devel- 
oped which  show  high  reliability  and  validity 
(see  123). 

Another  type  of  patient  measure  is  repre- 
sented by  "psychometric"  questionnaires  and 
personality  tests,  such  as  the  Minnesota  Multi- 
phasic Personality  Inventory  (MMPI).  Such  in- 
struments, which  are  perhaps  the  most  common 
type  of  measuring  tool  for  both  diagnostic  and 
assessment  purposes,  ask  a  variety  of  questions 
about  the  respondent's  thoughts.  Patients,  for 
example,  those  suffering  from  depression  and 
anxiety  (210),  have  been  shown  to  respond  to 
these  questions  according  to  particular  patterns. 
For  MMPI,  a  subset  of  questions  has  been  devel- 
oped which  tends  to  be  answered  differently  by 
those  who  are  trying  to  fake  responses  and  those 
who  are  not;  thus,  social  desirability  and  other 
biased  response  patterns  can  be  detected  (see 
55). 

Measures  From  Family  and  Friends 

Friends,  work  associates,  and  relatives  of  pa- 
tients are  often  asked  to  supply  information 
about  a  patient's  functioning,  and  such  data 
concerning  patient  behavior  and  inferred  mental 


states  have  been  used  to  assess  therapy  out- 
comes (e.g.,  116;  see,  in  particular,  a  report  by 
the  National  Institute  of  Mental  Health,  293). 
The  value  of  these  measures  may  be  due  to  the 
great  amount  of  information  that  family, 
friends,  and  work  associates  have  about  the 
functioning  of  the  patient.  These  individuals 
have  a  chance  to  observe  the  patient  in  a  variety 
of  situations  and  may  thus  have  a  great  deal  of 
"data"  on  which  to  base  their  responses  to  ques- 
tionnaires. Although  these  individuals  have  self- 
interests  which  can  bias  the  data,  they  may  have 
less  motivation  than  the  patient  to  reflect  social- 
ly desirable  responses  and  less  need  to  show 
gratitude  to  the  therapist. 

The  outcome  variables  included  on  these 
questionnaires  also  assess  outcomes  that  may  be 
more  important  to  "society"  than  the  variables 
included  on  patient  self-report  questionnaires. 
For  example,  questions  about  how  much  dis- 
turbance the  patient  causes  in  family  life  or 
work  routines,  or  how  often  the  patient  has 
secured  employment  or  performed  satisfactorily 
in  school  are  typically  included  in  these  instru- 
ments. Such  questions  may  be  more  important 
for  societal  evaluations  of  the  usefulness  of 
therapy  than  questions  about  coping  with  daily 
activities  that  are  typical  of  self-report  meas- 
ures. These  techniques  are  validated  both  by 
comparing  the  observations  of  those  who  know 
the  patient  against  one  another  (e.g.,  family 
members  compared  to  work  associates)  and  by 
comparing  these  data  with  other  available  in- 
formation about  the  patient. 

Measures  From  Therapists 

Measures  taken  from  the  therapist  and  others 
involved  in  the  therapeutic  process  are  another 
frequently  used  source  of  outcome  data.  While 
the  therapist  can  provide  a  first-hand  perspec- 
tive on  the  therapeutic  process,  such  data  may 
be  biased  because  the  therapist  has  a  vested  in- 
terest in  producing  positive  outcomes.  Never- 
theless, a  substantial  research  literature  on  such 
"clinical  judgments"  exists,  and  there  is  some 
evidence  that  therapists  can  provide  useful  and 
relatively  unbiased  reports.  Particularly  in 
terms  of  assessments  of  patient  functioning, 
there  is  evidence  that  therapists  can  provide 
valid  data  (e.g.,  67,108,161,184,198). 
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Generally,  the  more  specific  and  concrete  the 
observations  required  of  therapists,  the  greater 
the  resulting  interrater  reliability  and  validity 
(186,187).  These  observations  can  range  from 
"counts"  of  behavior  exhibited  by  patients  dur- 
ing therapy  to  therapist  ratings  of  the  patient's 
functioning  with  his/her  family  (based,  per- 
haps, on  how  the  patient  has  reported  family 
relationships).  To  assess  some  therapies,  where 
the  concern  is  more  with  mental  phenomena 
than  behaviors,  instruments  have  been  designed 
to  capture  nonobservable  behavior.  While  such 
instruments  can  provide  one  type  of  informa- 
tion about  the  effect  of  therapy,  their  useful- 
ness may  depend  on  how  the  goals  of  therapy 
have  been  described  and  the  availability  of 
validation  data. 

A  focus  on  the  specific  behaviors  of  the  pa- 
tient may  allow  therapists  to  provide  easily  vali- 
dated data  about  patient  functioning;  however, 
the  therapist's  desire  to  show  improvements  as  a 
result  of  therapy  may  bias  the  use  of  such  meas- 
ures. In  an  attempt  to  remove  this  bias  from 
functioning  judgments,  some  researchers  have 
trained  members  of  a  therapy  staff  who  are  not 
individuals  actually  involved  in  therapy  to  con- 
duct such  assessments  (e.g.,  209).  A  variety  of 
such  "blind"  data  collection  techniques  have 
been  employed. 

Measures  From  Community  Members 

Information  on  the  outcome  of  psychother- 
apy can  also  be  collected,  more  broadly,  from 
community  members  or  agencies.  These  meas- 
ures may  include  patient  data  on  criminal  arrest 
rates,  measures  related  to  the  patient's  work,  or 
rates  of  medical  utilization  (from  nonpsychiatric 
illness).  The  range  of  potential  community 
measures  is  very  large  and,  though  often  un- 
wieldy, seems  to  reflect  important  information 
that  is  needed  to  assess  adequately  the  effects  of 
psychotherapy.  As  is  described  later  in  the  dis- 
cussion of  cost-benefit  measurement  (ch.  5), 


such  variables  are  important  to  assess  in  order 
to  conduct  comprehensive  cost-effectiveness 
and  cost-benefit  analyses  (CEA/CBAs). 

Many  of  the  data  collected  on  community 
variables  can  be  assessed  in  relatively  direct 
ways.  The  information  often  can  be  gleaned 
from  existing  court  records,  hospital  charts,  in- 
surance claims,  and  similar  records.  To  be  uti- 
lized as  part  of  an  effectiveness  assessment, 
however,  such  measures  must  be  tested  for  relia- 
bility and  validity.  Oftentimes,  the  validity  of 
record  data  is  easy  to  establish  because  of  its  ob- 
vious relationship  to  desirable  psychotherapy 
outcomes  (referred  to  as  content  validity).  It 
should  be  noted  that  much  of  this  information 
can  also  be  collected  directly  from  patients  (e.g., 
data  on  medical  utilization,  if  it  can  be  shown 
that  patients  report  these  data  accurately). 

Community  data  have  also  been  taken  to 
mean  a  patient's  economic  contribution  to  his  or 
her  community  (see,  e.g.,  305).  Thus,  the  pa- 
tient's net  monetary  contribution  in  terms  of 
earnings  and  taxes  may  be  used  as  one  measure 
of  psychotherapeutic  outcome  (see  ch.  5).  These 
outcomes  have  been  assessed  for  psychotherapy 
in  a  number  of  investigations  (e.g.,  51,114)  and 
are  described  later  in  terms  of  CEA/CBAs  of 
psychotherapy. 

Summary 

The  outcomes  of  psychotherapy  can  be  meas- 
ured in  a  variety  of  ways.  It  is  probably  im- 
possible to  develop  any  single  measure  of  out- 
come which  would  reflect  the  diverse  changes 
that  might  be  brought  about  by  psychotherapy. 
At  present,  there  exists  a  diverse  set  of  pro- 
cedures for  eliciting  information  from  patients, 
family,  therapists,  and  others.  Data  from  each 
of  these  sources,  if  properly  assessed  for  reliabil- 
ity and  validity,  provide  information  needed  to 
assess  psychotherapeutic  effectiveness. 
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RESEARCH  DESIGNS 

The  development  of  reliable  and  valid  out- 
come measures  represents  only  one  part  of  the 
assessment  problem.  One  must  be  able  to  deter- 
mine which  of  the  many  processes  utilized  in 
therapy  are  responsible  for  improvements  and 
must  also  be  able  to  test  the  possibility  that  non- 
therapeutic  components  are  responsible  for  ob- 
tained effects.  The  particular  research  designs 
and  techniques  used  to  test  psychotherapeutic 
efficacy  are,  in  part,  determined  by  what  ques- 
tions are  being  asked.  These  questions  depend 
on  the  goals  of  both  the  patients  and  the  therapy 
program.  The  present  discussion  focuses  on  the 
possible  ways  of  carrying  out  psychotherapy 
outcome  research  and  how  questions  about  psy- 
chotherapy can  actually  be  tested. 

A  basic  element  of  good  research  design  is  to 
frame  the  questions  to  be  tested  in  a  very  spe- 
cific way.  Whereas  the  global  question  of  psy- 
chotherapy's  efficacy  may  be  obvious  (i.e.,  "is 
therapy  effective?"),  the  questions  asked  of 
research  need  to  be  more  circumscribed 
(21,207).  Such  circumscribed  questions  usually 
develop  as  research  progresses  through  a  series 
of  stages  (105).  At  a  formulative  stage,  research 
is  based  on  extensive  observation  and  summary 
descriptions  of  these  observations.  Then,  the 
focus  shifts  to  a  description  of  patterns  among 
data  elements.  At  a  later  stage,  explanations  and 
theories  of  the  observed  patterns  are  formed.  It 
is  usually  these  later  explanations  that  are  tested 
in  formal  experiments  and  that  represent  the 
bulk  of  the  outcome  research  literature.  Al- 
though effectiveness  studies  can  be  conducted 
on  a  post  hoc  basis  (i.e.,  a  design  constructed 
after  the  data  have  been  collected),  such  re- 
search usually  can  be  interpreted  only  when 
theory  and  formal  experimental  evidence  are 
available  (e.g.,  47,151). 

The  purpose  of  the  present  report  is  to  under- 
stand psychotherapeutic  effectiveness;  thus,  the 
focus  is  primarily  on  data  collected  in  actual 
treatment  settings  (in  vivo).  A  great  deal  of  psy- 


chotherapy research  has  been  carried  out  under 
laboratory  or  analog  conditions,  however,  and 
at  least  some  researchers  (e.g.,  10)  view  the  data 
from  this  research  as  very  important.  To  the  ex- 
tent that  such  data  provide  theoretical  support 
for  in  vivo  findings,  they  are  probably  neces- 
sary to  consider.  OTA,  in  previous  discussions 
of  the  evaluation  of  medical  technology  (202), 
has  regarded  this  as  efficacy  rather  than  effec- 
tiveness research  (ideal  v.  actual  conditions). 
When  applied  to  psychotherapy,  however,  the 
efficacy/effectiveness  distinction  seems  ambigu- 
ous because  of  the  variety  of  factors  (therapist, 
patient,  setting)  which  affect  the  outcome  of  a 
particular  treatment  and  the  absence  of  a  clear 
demarcation  between  laboratory  and  nonlabo- 
ratory  conditions.  It  seems  desirable  in  the  case 
of  psychotherapy,  instead  of  trying  to  differenti- 
ate between  assessments  of  efficacy  and  effec- 
tiveness research,  to  note  clearly  the  different 
conditions  under  which  research  is  conducted. 

In  terms  of  developing  research  tests  of  thera- 
peutic effects,  some  limitations  of  prevailing  sci- 
entific logic  should  also  be  noted  (see  41).  Test- 
ing particular  hypotheses  does  not  permit  the 
psychotherapy  researcher  to  prove  that  a  partic- 
ular therapeutic  effort  causes  a  particular  out- 
come; instead,  one  tests  whether  alternative  ex- 
planations can  be  disregarded  (47,105).  Result- 
ing inferences  are  probabilistic  and  indicate, 
within  identifiable  error  rates,  the  likelihood 
that  generalization  can  be  made  from  the  study's 
sample  to  other  populations. 

The  inherent  variability  of  human  behavior, 
thoughts,  and  feelings  often  makes  the  findings 
of  psychotherapy  research  equivocal.  A  great 
deal  of  variability  will  exist  under  any  condi- 
tions, and  these  variables  must  be  separated 
from  the  effects  of  the  treatment.  In  addition, 
because  patients  can  be  affected  by  many  vari- 
ables (i.e.,  factors  other  than  psychotherapy) 
that  cannot  be  controlled,  there  may  be  numer- 


68-094  0-80-5 


26  •  Background  Paper  #3:  The  Efficacy  and  Cost  Effectiveness  of  Psychotherapy 


ous  alternative  explanations  for  any  apparent 
improvements  produced  by  psychotherapy.  The 
number  of  alternative  explanations  can  be  re- 
duced somewhat  by  careful  design  of  psycho- 
therapy research  and  by  the  inclusion  of  control 
conditions  that  hold  all  elements  constant,  ex- 
cept some  aspect  of  a  treatment.  A  variety  of  ex- 
perimental designs  have  been  developed  to  re- 
duce the  plausibility  of  a  number  of  standard 
alternative  explanations.  These  research  designs 
are  not  unique  to  psychotherapy  (see  27,41, 
94,165),  but  their  use  in  assessing  the  efficacy  of 
psychotherapy  raises  a  significant  set  of  unique 
problems.  Several  types  of  research  designs  that 
can  be  used  to  assess  psychotherapy  are  de- 
scribed below. 

Therapy  Versus  No-Therapy  Designs 

The  classical  research  design,  as  applied  to 
psychotherapy,  assigns  patients  either  to  receive 
psychotherapy  or  not  to  receive  psychotherapy 
according  to  a  random  selection  procedure.  It 
can  be  expected  that  randomization  will  distrib- 
ute differences  in  patient  characteristics  (e.g., 
level  of  mental  dysfunction,  amenability  to 
treatment)  equally  between  the  psychotherapy 
and  no-psychotherapy  conditions.  Typically, 
the  psychotherapy  condition  is  referred  to  as  the 
"experimental"  condition,  while  the  no-psycho- 
therapy condition  is  referred  to  as  the  "control" 
condition.  The  functioning  of  each  subject  in  the 
experiment  is  measured  following  psychother- 
apy (or  following  an  equal  interval  of  time  for 
control  subjects).  Although  measures  can  be 
taken  at  other  times  (e.g.,  pretherapy),  addi- 
tional design  problems  (having  to  do  with  the 
effect  of  taking  a  test  or  responding  to  a  ques- 
tionnaire more  than  once)  result  with  such 
procedures. 

The  measurements  of  functioning  (including 
cognitive  and  behavioral  outcomes)  obtained 
from  the  treatment  condition  contain  three  pos- 
sible "effects:"  1)  "effects"  of  treatment,  2)  "ef- 
fects" due  to  whatever  nontherapy  factors  are 
affecting  the  patient,  and  3)  "effects"  of  the 
haphazard  fluctuations  in  functioning  measures 
caused  by  imperfections  in  measurement  instru- 
ments and  the  variability  of  behavior.  The 
measurements  of  patient  functioning  obtained 


from  the  control  condition  are  used  to  "sub- 
tract" the  "effects"  due  to  treatment  from  the 
"effects"  due  to  other  factors.  A  control  condi- 
tion is  necessary  to  perform  the  above  "subtrac- 
tion" because  it  provides  the  only  empirical  way 
of  knowing  how  nontherapy  factors  and  meas- 
urement problems  affected  the  outcome. 

When  this  type  of  therapy  versus  no-therapy 
research  design  is  used,  explanations  of  appar- 
ent improvements  in  patient  functioning  that 
are  actually  due  to  factors  other  than  therapy 
itself  can  be  rejected  with  reasonable  confi- 
dence. As  long  as  patients  have  been  assigned 
randomly  to  therapy  and  no-therapy  condi- 
tions, it  can  be  assumed  (within  identifiable 
probability  limits)  that  the  obtained  "effects" 
are  due  to  the  psychotherapeutic  treatment.  If 
patients  are  not  randomly  assigned,  but  are 
"matched"  on  various  characteristics,  such  an 
unequivocal  statement  is  not  possible.  It  can  be 
argued  that  differences  existed  between  experi- 
mental and  control  group  subjects  that  were  not 
controlled  by  matching  and  that  these  charac- 
teristics are  responsible  for  differences  obtained 
between  therapy  and  no-therapy  groups.  The 
absence  of  a  control  group  makes  such  in- 
ferences about  causal  factors  extremely  difficult 
to  develop. 

Therapy  Versus  Therapy 

The  basic  rationale  used  to  distill  the  effects  of 
psychotherapy  from  effects  of  measurement  and 
factors  unrelated  to  therapy  can  be  extended  to 
test  for  the  superiority  of  one  psychotherapy 
over  another.  In  such  a  therapy  comparison 
study,  patients  are  assigned  randomly  to  ther- 
apy A,  therapy  B,  .  .  .,  and,  perhaps,  to  a  no- 
therapy  or  delayed-therapy  group.  In  effect, 
such  a  design  results  in  the  use  of  multiple  treat- 
ment groups.  The  use  of  no-treatment  control 
groups  is  not  ruled  out,  but  such  groups  are 
often  not  employed  in  therapy  comparison  stud- 
ies because  the  purpose  of  such  studies  is  the 
assessment  of  the  best  therapy.  It  should  be 
noted  that  comparisons  across  therapies  are 
often  made  without  use  of  an  experimental  re- 
search design.  When  that  is  done,  patients  who 
have  received  different  therapies  are  compared 
without  regard  to  the  selection  factors  that  in- 
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fluenced  which  patients  received  which  therapy. 
In  such  nonexperimental  research,  differences 
may  be  due  to  a  variety  of  factors  (e.g.,  pre- 
existing differences  between  patients  in  each 
group),  and  these  variables  need  to  be  con- 
trolled before  such  data  are  useful. 

In  another  common  design  used  to  compare 
therapies,  two  potentially  effective  therapy 
techniques  are  presented  separately  to  two 
groups  of  subjects  and  in  combination  to  a  third 
group  (yielding  individual  and  combination 
therapy  conditions).  Statistical  analyses  are 
used  to  separate  the  effects  of  the  therapies  and 
to  compare  them  with  no-therapy  conditions.  A 
particular  concern  of  the  statistical  analysis  is  to 
test  for  interaction  effects  (e.g.,  where  therapies 
combine  to  produce  an  effect  that  is  different 
from  the  sum  of  the  two  effects  alone).  A  vari- 
ant of  this  type  of  design  has  been  used  to  assess 
the  joint,  as  well  as  separate,  effects  of  chemo- 
therapy and  psychotherapy  (e.g.,  142). 

Therapy  Versus  "Placebo"  Therapy 

One  problem  that  has  plagued  much  psycho- 
therapy research  (as  well  as  other  research  on 
medical  interventions)  is  that  some  of  the  effects 
obtained  by  psychotherapy  researchers  may  be 
due  to  placebo  effects.  The  "aura"  of  being  in 
therapy  and  the  expectancy  that  one  is  finally 
about  to  be  "cured"  may  be  a  form  of  treatment 
(136,256).  The  problem  of  separating  these  ef- 
fects from  those  of  formal  therapies  is  analogous 
to  the  use  of  sugar  pills  in  controlled  medical 
research  and  involves  the  use  of  placebo-control 
conditions.  In  such  conditions,  the  patient  may 
receive  attention  from  a  therapist,  but  therapeu- 
tically meaningful  discussion  is  avoided  and  no 
specific  techniques  are  used  (e.g.,  176).  It  is 
easier  to  employ  such  control  procedures  when 
testing  the  efficacy  of  behavioral,  or  even  psy- 
chodynamic,  therapies,  in  which  the  therapist 
plays  an  active  and  directive  role. 

Probably,  because  of  the  nature  of  psycho- 
therapy and  the  difficulty  of  specifying  the 
precise  ingredients  of  therapy,  it  is  impossible  to 
control  all  placebo  effects  (207,280).  The  rela- 
tionship that  a  therapist  establishes  with  a  pa- 
tient is  acknowledged  (e.g.,  84)  to  be  an  impor- 
tant component  of  therapy,  and  it  is  not  clear 


how  such  effects  should  be  distinguished  from 
treatments  per  se.  It  should  also  be  recognized 
that  while  placebo  effects  may  inflate  the  true  ef- 
fects of  therapy,  they  may  often  distort  the  data 
obtained  in  control  conditions.  Few  control  con- 
ditions can  be  "pure"  in  the  sense  that  patients 
do  not  interact  with  a  therapist.  If  for  no  other 
reason  than  to  monitor  the  patient,  those  in  con- 
trol conditions  must  usually  be  supervised  by  a 
therapist.  The  effects  of  this  supervision  may 
make  therapy  appear  to  be  less  effective. 

Control  conditions  may  be  introduced  to 
assess  the  effects  of  such  factors  as  therapist  "de- 
mand characteristics."  An  ardent  researcher  or 
therapist  may  communicate  to  patients  what  the 
results  of  an  experiment  are  "expected"  to  be 
(e.g.,  240).  Experiments  using  self-report  meas- 
ures obtained  from  patients  are  especially  prone 
to  this  problem.  To  examine  the  strength  of 
these  demand  characteristics,  psychotherapy 
researchers  can  use  designs  in  which  patients  are 
told  that  therapy  will  produce  a  temporary 
worsening  of  functioning,  when  in  actuality  the 
researcher  expects  no  worsening.  Alternatively, 
the  researcher  may  tell  patients  in  some  condi- 
tions that  therapy  should  not  be  effective  for 
several  months,  when  gradual  improvement  ac- 
tually is  expected.  If  the  "demanded"  effects  are 
not  found  in  these  conditions,  the  researcher  can 
have  some  confidence  that  demand  character- 
istics are  not  causing  therapy  effects.  Although 
these  controls  may  not  be  employed  in  every 
outcome  study,  they  are  often  used  in  develop- 
ing research  designs. 

Quasi-Experimental  Designs 

For  a  variety  of  reasons,  including  ethical 
problems  of  withholding  treatment  (see  discus- 
sion below),  the  experimental  designs  described 
above  are  often  not  employed.  The  real  or  per- 
ceived difficulties  of  assigning  patients  on  a  ran- 
dom basis  to  therapy  or  no-therapy  conditions 
make  other  types  of  comparisons  necessary. 
There  are  a  number  of  quasi-experimental  de- 
signs that  can  be  used  in  such  circumstances  (see 
41).  Such  designs  are  sometimes  considered  to 
be  poor  substitutes  for  "pure"  experimental 
designs,  but  this  may  not  be  the  best  way  to 
view  them.  Quasi-experimental  designs,  if  care- 
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fully  constituted,  eliminate  the  most  important 
plausible  alternative  explanations  for  the  results 
of  an  experiment  and  can  provide  useful  infor- 
mation. Quasi-experiments  based  on  these 
designs,  however,  may  involve  complex  statis- 
tics and  require  the  collection  of  many  more 
data  (perhaps  from  fewer  subjects)  than  a  "true" 
experiment  would. 

One  common  quasi-experimental  design  in 
psychotherapy  is  referred  to  as  an  intensive 
design  (97).  Typically,  intensive  designs  use 
fewer  subjects  than  experimental  designs,  but 
compensate  by  obtaining  more  measures  on  a 
frequent  basis  and  by  obtaining  clear  "baseline" 
measures  of  patient  functioning.  The  intent  is 
that  sudden  improvements  will  be  dramatic  and 
closely  correlated  with  the  onset  of  therapy.  To 
the  extent  that  these  changes  are  sudden,  many 
of  the  alternative  explanations  can  be  ruled  out 
(despite  the  lack  of  a  control  group).  The  use  of 
intensive  designs,  however,  depends  on  the  type 
of  therapy  and  the  outcomes  that  are  expected. 
Only  therapies  that  posit  observable  and  rapid 
changes  (typically,  the  more  behavioral  thera- 
pies) can  be  tested  with  such  designs. 

Return-to-Baseline  Designs. — In  this  design, 
patient  functioning  is  measured  several  times 
before  therapy  (the  baseline  period)  and  several 
times  during  therapy  (the  manipulation  period). 
Therapy  is  then  withdrawn  abruptly  (reversal  to 
second  baseline),  and  it  is  expected  that  there 
will  be  a  return  to  lower  levels  of  functioning 
similar  to  those  found  during  the  first  baseline 
period.  Repeated  measurement  continues  during 
this  period  and  during  a  return  to  therapy 
(second  manipulation).  Therapy  is  then  phased 
out  with  the  hope  that  improvements  will  be 
maintained. 

An  example  of  this  type  of  design  is  Allen, 
Hart,  Buell,  Harris,  and  Wolf's  (3)  study  of  a 
single  child.  The  child  had  isolated  herself  from 
other  children,  causing  severe  psychological 
problems.  The  amount  of  time  that  the  child 
spent  with  adults  and  other  children  was  meas- 
ured during  the  baseline  period.  A  simple  ther- 
apy that  provided  reinforcement  for  her  inter- 
actions with  other  children  was  then  provided. 
Measurement  continued  during  this  period,  as 


well  as  after  therapy.  Later,  there  was  an  abrupt 
withdrawal  of  the  reinforcement  (reversal  to 
second  baseline),  followed  by  its  reinstitution. 
The  results  showed  very  noticeable  changes  in 
the  girl's  interactions.  In  the  absence  of  other 
factors  (besides  therapy),  the  effects  were  seen 
as  demonstrating  therapeutic  efficacy. 

Intrasubject  Multiple  Baseline  Designs. —  To 

obtain  information  analogous  to  that  obtained 
from  a  separate  no-therapy  control  group,  it  is 
also  possible  to  separately  treat  and  assess  func- 
tioning in  different  areas  of  the  patient's  life. 
Thus,  treatment  is  designed  to  improve  one 
aspect  of  functioning,  and  the  patient's  other 
behavior  is  used  as  a  control.  Such  a  design, 
while  having  advantages  because  changes  can 
be  compared  with  only  one  subject,  suffers  from 
the  obvious  problem  that  effects  may  generalize 
across  function  areas.  In  fact,  for  some  therapies 
generalization  of  effects  could  be  expected,  and 
an  intrasubject  multiple  baseline  design  would 
lead  to  "missing"  the  effects  of  therapy. 

An  example  of  this  type  of  design  is  pro- 
vided by  Morganstern  (189).  The  effectiveness 
of  aversive  conditioning  on  obesity  (pairing 
nauseous  smoke  with  eating  certain  foods)  was 
examined  by  conditioning  a  patient  to  one  type 
of  food,  then  after  several  days,  to  another.  Be- 
cause eating  of  other  foods  was  found  not  to 
change  until  aversive  conditioning  was  applied 
to  the  particular  foods,  it  was  possible  to  infer 
that  the  therapy  produced  changes. 

There  are  a  number  of  variations  on  this 
intrasubject  multiple  baseline  design.  Multiple 
subjects  can  be  used,  and  different  combinations 
of  treatment  can  be  tested.  This  design  does  not 
eliminate  alternative  explanations  due  to  place- 
bo or  demand  characteristics,  but  can  eliminate 
some  problems  inherent  in  nonrandomized  ex- 
periments. Thus,  additional  subjects  can  serve 
as  controls  to  assess  the  importance  of  factors 
such  as  spontaneous  remission  (improvement 
without  treatment).  However,  the  use  of  this 
design  may  be  severely  limited  by  the  types  of 
problems  (very  specific)  and  the  therapies 
(mostly  behavioral)  which  one  tests. 
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Program  Evaluations 

The  design  considerations  described  above 
implicitly  refer  to  psychotherapy  as  a  unitary 
treatment  that  can  be  applied  or  not  applied 
to  form  experimental  and  control  conditions. 
However,  as  chapter  2  points  out,  psychother- 
apy treatments  are  comprised  of  a  number  of 
factors.  The  theoretical  orientation  of  the  ther- 
apy, usually  the  basis  of  a  label  for  the  treat- 
ment, may  not  adequately  describe  the  treat- 
ment as  it  is  actually  delivered.  Depending  on 
the  nature  of  patients'  problems,  therapists'  ori- 
entation and  skill,  as  well  as  aspects  of  the  deliv- 
ery setting,  therapy  may  have  different  out- 
comes. One  possibility  is  to  treat  each  of  these 
factors  as  an  independent  variable  and  to  con- 
struct multifactor  designs.  In  such  designs,  sub- 
jects are  randomly  assigned  to  different  thera- 
pies, therapists,  and  settings.  A  more  recent 
trend,  though,  has  been  to  conduct  such  out- 
come research  through  program  evaluations 
(see  212).  In  such  evaluative  research  (e.g.,  9, 
159,252),  one  evaluates  a  complex  of  treatment 
variables  that  have  been  organized  as  a  program 
(e.g.,  a  community  mental  health  center 
(CMHC)). 

Thus,  for  example,  a  program  evaluation 
study  of  a  CMHC  tries  to  assess  how  and  to 
what  extent  patients  who  receive  treatment  at 
the  center  are  aided.  The  CMHC,  in  addition  to 
being  community  based  (which  is  hypothesized 
to  be  an  adjunct  to  treatment),  may  offer  pa- 
tients a  number  of  therapies,  and  patients  may 
be  treated  by  multiple  professionals  and  para- 
professionals.  Under  such  circumstances,  where 
joint  effects  of  these  treatments  are  expected  and 
where  it  is  extremely  difficult  to  separate — for 
research  design  purposes — the  components  of 
treatment,  program  evaluation  yields  a  design 
that  may  be  more  compatible  with  the  actual 
circumstances.  Evaluative  research  does  not 
preclude  the  conduct  of  experiments  where  in- 
dividual aspects  of  the  treatment  are  assessed. 

The  designs  for  program  evaluation  studies 
of  psychotherapy  can  include  aspects  of  the 
true  and  quasi-experimental  designs  described 
above.  In  general,  the  same  methodological 
considerations  for  research  designs  apply  to 


program  evaluations  (229,298,303).  There  is,  in 
fact,  a  substantial  literature  on  the  use  of  the  ex- 
perimental designs  in  program  evaluation  (e.g., 
230,249).  The  literature  describes  both  how 
complex  variables  such  as  psychotherapy  can  be 
conceptualized  and  the  conditions  for  imple- 
menting randomized  designs.  Similarly,  the 
literature  on  quasi-experimental  designs  has 
been  related  to  program  evaluation  (e.g.,  41, 
230).  The  principal  difference  between  these  de- 
signs and  traditional  research  design  is  in  how 
one  conceptualizes  the  treatment.  In  a  program 
evaluation  study,  the  treatment  includes  a  num- 
ber of  elements.  These  elements,  at  least  in  the 
initial  stages  of  such  an  evaluation,  are  not  sepa- 
rately tested. 

Even  though  program  evaluations  are  de- 
signed specifically  to  aid  in  policy  decisionmak- 
ing, there  are  a  number  of  endemic  problems. 
Just  as  it  is  difficult  to  organize  a  traditional  psy- 
chotherapy outcome  study  (i.e.,  randomly  or 
otherwise  assign  patients  to  treatments),  it  is  dif- 
ficult to  organize  program  evaluation  studies.  In 
fact,  because  the  randomization  units  are  more 
complex,  such  evaluation  studies  are  often  very 
difficult  to  carry  out  well,  and  there  is  a  sub- 
stantial literature  about  implementation  fail- 
ures. In  addition,  program  evaluation  studies 
may  not  resolve  the  underlying  conceptual 
problems  involved  in  assessing  psychotherapy. 
It  may  be  difficult  to  determine  through  a  pro- 
gram evaluation  what  factors  were  responsible 
for  the  success  or  failure  of  the  program. 

Difficulties  in  Conducting  Research 

There  are  a  variety  of  problems  which  make  it 
difficult  to  conduct  psychotherapy  outcome  re- 
search. Some  of  these  problems,  which  relate  to 
the  inappropriateness  of  some  methods  to  test 
particular  therapies  (e.g.,  intensive  designs)  and 
the  problem  of  multiple  factors  affecting  out- 
come (e.g.,  the  role  of  therapist  variable),  have 
already  been  described.  There  are  also  prob- 
lems, however,  which  are  perhaps  more  impor- 
tant, having  to  do  with  the  pragmatic  and  eth- 
ical difficulties  of  conducting  experimental 
research. 
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While  the  advantages  of  experimental  meth- 
ods to  develop  unequivocal  data  are  well  known 
and  widely  accepted,  there  are  obvious  ethical 
problems  connected  with  decisions  to  withhold 
treatment  (e.g.,  45,229)  or  to  make  treatment 
available  on  a  random  basis.  Especially  if  one 
believes  that  therapy  is  efficacious,  it  is  dif- 
ficult not  to  allow  certain  patients  to  receive 
treatment  for  research  purposes.  Several  consid- 
erations are  important  in  thinking  about  this 
dilemma. 

One  important  consideration  is  the  necessity 
of  conducting  research,  especially  that  which 
employs  randomized  control  groups.  Such  stud- 
ies (which,  in  medicine,  are  referred  to  as  ran- 
domized clinical  trials)  make  it  possible  to  assess 
causality  in  the  most  unambiguous  way  (see, 
e.g.,  36,41,165).  Although,  from  a  theoretical 
point  of  view,  the  value  of  randomized  control 
group  studies  has  long  been  recognized,  there  is 
now  some  evidence  that  such  studies  better  en- 
able researchers  to  detect  inefficacious  treat- 
ments (see,  e.g.,  94).  If  less  adequate  designs  are 
used  (i.e.,  designs  not  employing  randomized 
control  conditions),  decision  errors  may  result. 
Thus,  it  may  appear,  perhaps  because  of  the  ef- 
fects of  other  variables,  that  the  treatment  is  ef- 
fective when  it  is  not. 

One  resolution  to  the  ethical  problems  pre- 
sented by  needing  to  withhold  treatment  is  sug- 
gested by  the  nature  of  the  dilemma.  If  a  treat- 
ment has  not  been  demonstrated  to  be  effective, 
then  it  may  not  be  unethical  to  deny  treatment 
to  some  individuals.  The  treatment,  in  the 
absence  of  empirical  data,  may  not  be  accom- 
plishing anything.  Actual  treatment  providers 
(i.e.,  therapists)  may  not  share  this  view  and, 
therefore,  may  be  reluctant  to  participate  in  this 
form  of  experimental  research.  At  the  very 
least,  practitioners  might  want  to  be  able  to 
supervise  control  group  subjects  and,  in  effect, 
provide  partial  treatment. 

One  pragmatic  resolution  to  the  ethical  dilem- 
ma of  this  research  is  to  provide  control  group 
subjects  with  access  to  treatment  once  the  ex- 
periment has  been  concluded.  This  is  often  re- 
ferred to  as  the  "waiting  list"  approach.  It  is 
made  a  more  attractive  option  in  some  studies 
(which  compare  several  therapeutic  approaches) 


by  offering  the  delayed  control  group  subjects 
the  form  of  treatment  which  the  experiment 
demonstrated  as  best.  Obviously,  this  option 
can  be  provided  only  when  therapy  is  of  a  rela- 
tively short  duration.  It  is,  however,  easy  to  im- 
plement when  resources  are  very  limited.  If  only 
a  small  group  of  patients  can  be  given  treatment 
at  any  one  time,  a  waiting  list  (established  by 
random  assignment)  may  be  practicable  (al- 
though it  might  be  viable  only  when  patients  do 
not  have  other  treatment  options). 

Whatever  the  justification  for  random  assign- 
ment to  treatment  and  control  conditions,  ex- 
plicit guidelines  (established  by  the  Department 
of  Health  and  Human  Services  and  professional 
societies)  must  be  followed  to  protect  patients' 
rights.  One  central  principle  of  these  guidelines 
is  that  participation  be  voluntary,  based  on  pa- 
tients' (or  legal  surrogates')  "informed  consent" 
(see,  e.g.,  6).  Informed  consent  procedures  re- 
quire that  subjects  be  informed  as  to  the  pur- 
poses of  the  experiment,  known  risks,  data  to  be 
collected,  and  how  the  data  will  be  used.  Sub- 
jects also  have  the  right  to  leave  the  experiment 
at  any  time  and,  usually,  must  be  told  of  other 
treatment  resources.  The  procedures  also  re- 
quire that  review  panels  be  established  to  ap- 
prove research  protocols  and  monitor  the  con- 
duct of  this  human  research. 

Although  such  procedures  may  create  differ- 
ential attrition  across  conditions  of  an  experi- 
ment, and  perhaps  result  in  only  certain  types  of 
patients'  or  treatments'  being  involved  in  re- 
search, it  does  not  rule  out  the  use  of  experimen- 
tal designs.  Under  a  variety  of  circumstances, 
individuals  will  agree  to  participate  in  experi- 
mental research,  and  the  problems  may  have 
more  to  do  with  the  researchers  or  therapists  in- 
volved than  with  patients.  As  noted  above,  until 
unequivocal  evidence  is  available  about  psycho- 
therapy and  more  resources  are  available  for 
such  treatments,  it  seems  necessary  to  conduct 
such  experimental  studies.  These  studies  are 
critical  to  providing  unequivocal  tests  of  the 
theoretical  hypotheses  suggested  by  basic  re- 
search. For  many  therapies,  without  experi- 
mental evidence,  there  will  be  no  way  of  resolv- 
ing questions  about  alternative  explanations  for 
their  effects  (see  207).  In  order  to  carry  out  such 
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studies,  it  may  be  necessary  for  the  Government 
to  develop  special  rules  for  their  conduct.  Thus, 
for  example,  special  reimbursement  procedures 
may  need  to  be  applied  so  that  treatment  and  re- 
search costs  can  be  separated. 

Given  the  difficulties  of  conducting  in  vivo 
experimental  research,  it  is  perhaps  not  surpris- 
ing that  much  of  the  best  controlled  research  has 
been  laboratory  analogs  to  clinical  settings.  In 
these  analog  settings,  psychotherapy  is  offered 
to  patients  who  often  have  less  severe  dysfunc- 
tions than  are  seen  in  psychotherapy  clinics. 
However,  these  patients  can  be  assigned  to  no- 
treatment  and  placebo  conditions  because  the 
psychotherapy  researcher  controls  the  program 
and  because  the  loss  of  the  therapeutic  benefit 
has  less  impact.  This  type  of  research  may  yield 


more  rigorous  experimental  findings,  but  has 
low  external  validity;  that  is,  the  patients  (often 
undergraduate  students),  therapists  (often  doc- 
toral students),  and  procedures  (more  theoreti- 
cally oriented  and  less  eclectic  than  is  typical) 
are  not  representative  of  those  used  in  "real" 
psychotherapy  (e.g.,  162,169).  It  should  be 
noted  that  some  researchers  suggest  that  this  is 
the  best  way  to  conduct  rigorous  investigations 
of  the  effectiveness  of  psychotherapy  and  that 
the  innovative  techniques  developed  and  tested 
in  such  settings  can  be  transferred  to  more 
"messy,"  real  world  settings  (10,135,136).  In 
essence,  they  argue  that  the  differences  between 
actual  and  analog  settings  are  not  as  great  as 
might  be  anticipated  and  that  generalizability 
can  be  demonstrated  for  these  analog  studies  (at 
least  for  some  therapies  and  conditions  of 
treatment). 


INTEGRATING  FINDINGS  FROM  PSYCHOTHERAPY  RESEARCH 


Although  the  key  to  evaluating  the  efficacy  of 
psychotherapy  lies  in  the  conduct  of  well- 
designed  research  which  uses  multiple  measures 
that  are  both  reliable  and  valid,  this  may  not  be 
sufficient  for  policymaking  purposes.  No  one 
research  design  (including  true  experiments  and 
program  evaluations)  will  enable  the  develop- 
ment of  definitive  information  about  the  effects 
of  psychotherapy,  nor  will  the  measures  used  in 
any  one  study  be  adequate  for  all  purposes.  In 
part,  this  is  because  of  the  current  state  of 
theorizing  about  psychopathology,  which  en- 
compasses a  number  of  approaches  each  with 
different  ideas  about  research.  The  problems, 
however,  are  not  only  theoretical,  but  also  re- 
flect the  limitations  of  the  scientific  process  with 
respect  to  developing  unambiguous  conclusions 
on  the  basis  of  individual  research  studies. 
Given  the  diversity  of  criteria  against  which 
such  psychotherapy  research  must  be  evaluated, 
as  well  as  the  divergent  theoretical  views, 
methods  are  required  to  synthesize,  aggregate, 
or  integrate  the  findings  of  multiple  studies. 

Despite  the  fact  that  it  might  be  desirable  to 
have  a  few  studies  which  "settle"  the  effec- 
tiveness question,  for  the  most  part,  it  is  neces- 


sary to  treat  cautiously  the  results  of  individual 
studies.  Their  results  must  be  judged  against 
other  research  designed  to  test  the  same  or  simi- 
lar hypotheses.  Traditionally,  such  judgments 
have  been  made  through  literature  reviews, 
where  scholars  analyze  a  body  of  research. 
These  scholarly  analyses  are  reviewed  by  peers 
and  published  so  that  other  researchers  can 
comment.  Because  of  the  scientific  standards 
underlying  peer  review  (see,  e.g.,  39),  most  such 
reviews  reflect  honest  efforts  to  weigh  the  ap- 
propriate evidence. 

However,  even  though  the  evidence  may  be 
reported  accurately,  it  is  fairly  easy  to  be  selec- 
tive about  the  research  data  one  includes.  It  is 
rarely  possible,  especially  when  considering  the 
host  of  problems  for  which  psychotherapy  has 
been  applied,  to  include  all  potentially  relevant 
research.  Any  researcher  who  conducts  a  review 
makes  a  number  of  implicit  and  explicit  choices 
about  what  will  be  included.  As  well,  choices 
are  made  about  what  elements  of  the  studies  will 
be  discussed  and  those  that  will  be  ignored;  in 
effect,  the  author  "slants"  the  review  to  support 
a  particular  hypothesis  or  viewpoint.  Clear  cri- 
teria are  not  always  provided  for  judging  the 
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methodological  adequacy  of  the  studies  in- 
cluded in  the  review. 

To  make  this  process  more  scientific,  or  at 
least  more  systematic,  a  number  of  procedures 
have  recently  been  developed  and  applied  to 
assessments  of  the  psychotherapy  outcome  liter- 
ature. These  procedures,  although  they  have 
their  own  limitations  and  are  by  no  means 
unanimously  accepted,  represent  attempts  to 
make  sense  of  the  burgeoning  and  often  contra- 
dictory research  about  psychotherapy.  Two 
such  procedures  are  discussed  below:  "box- 
score  analyses"  and  "meta-analyses." 

Box-Score  Analyses. — The  procedure  of 
"box-score  analysis"  begins  with  identification 
of  a  population  of  research  studies  (see,  e.g., 
162).  Usually,  the  reviewer  establishes  certain 
standards  and  excludes  studies  that  are  not  suffi- 
ciently rigorous  by  methodological  criteria,  or 
are  otherwise  not  appropriate.  The  latter  cate- 
gory might  include  studies  that  are  well  de- 
signed but  lack  sufficiently  reliable  or  valid  out- 
come measures.  It  might  also  include  studies 
that  do  not  have  patients  assigned  randomly  to 
conditions.  The  difference  between  this  proce- 
dure and  that  used  in  literature  reviews  is  not 
sharply  defined  and  is  only  the  greater  degree  to 
which  criteria  are  explicitly  described  in  box- 
score  analyses  and  used  to  select  research  for  the 
review. 

All  studies  meeting  the  reviewer's  criteria  are 
culled  from  some  defined  set  of  sources  (e.g., 
journals)  and  are  then  sorted  into  categories. 
Typical  categories  might  be  dysfunction  treated, 
therapy  technique,  and/or  training  of  therapist. 
Finally,  the  reviewer  evaluates  each  study's  out- 
come (e.g.,  "yes,"  "no,"  "equivocal")  and  tallies 
scores  using  the  predefined  categories  (e.g.,  dys- 
function, technique,  therapists). 

This  method  of  categorization  and  then  tally- 
ing of  effects  is  designed  to  systematize  the  lit- 
erature review  process.  However,  it  still  leaves  a 
great  deal  of  room  for  individual  judgment  and 
typically  uses  a  rather  simplistic  measure  of 
treatment  outcome.  Importantly,  it  does  not 
take  account  of  the  strength  of  findings  within 
particular  studies.  A  box-score  analysis  does  not 
take  account  of  the  methodological  rigor  of  a 
study,  except  in  global  fashion  (e.g.,  by  exclud- 


ing studies  without  a  particular  kind  of  control 
condition).  It  is,  nevertheless,  a  useful  proce- 
dure, and  a  number  of  important  box-score-type 
reviews  of  the  psychotherapy  literature  are  re- 
ported in  chapter  4. 

Meta- Analyses. — As  described  by  Smith  and 
Glass  (267)  and  Smith,  Glass,  and  Miller  (268), 
"meta-analysis"  is  a  procedure  for  integrating 
research  findings  (see  also  239).  It  is  a  more  re- 
cent, and  undoubtedly  controversial,  set  of  pro- 
cedures (see  ch.  4).  Meta-analysis  employs  sta- 
tistical techniques  for  aggregating  data  and  for 
determining  relationships  between  causal  varia- 
bles and  outcomes.  Usually,  the  first  step  in  a 
meta-analysis  is  the  precise  description  of  a  pop- 
ulation of  studies  on  which  the  analyses  will  be 
based.  In  this  respect,  a  meta-analysis  is  con- 
ducted similarly  to  a  box-score  analysis. 

In  a  meta-analysis,  however,  studies  are  then 
coded  on  a  set  of  variables  that  are  thought  to  be 
related  to  outcomes — the  number  of  these  vari- 
ables is  at  the  discretion  of  the  analyst.  The  ex- 
perience of  the  therapist,  the  patients'  symp- 
tomatology, the  quality  of  the  research  design, 
and  the  setting  of  the  treatment  are  examples  of 
the  categories  of  variables  that  would  be  coded 
in  an  analysis  of  psychotherapy.  These  meas- 
ures are  later  correlated  with  the  outcomes 
(usually  quantified)  and  used  as  the  basis  for 
organizing  outcome  results  in  terms  of  aspects 
of  the  studies.  In  Smith,  et  al.'s  (268)  work,  an 
outcome  measure  for  psychotherapy  based  on 
the  size  of  the  effect  (standardized)  was  com- 
puted for  almost  500  studies.  The  studies  came 
from  a  population  of  controlled  (i.e.,  compari- 
son group)  investigations.  Although  a  global  ef- 
fect size  measure,  as  calculated  by  Smith,  et  al., 
does  not  differentially  weigh  studies  according 
to  the  quality  of  the  measures  employed  or  the 
design,  such  factors  are  controlled  by  coding 
each  study  in  terms  of  instrument  and  design 
validity.  If,  for  example,  there  is  a  difference  be- 
tween the  size  of  the  effects  in  studies  that  used 
poor  measures  and  those  that  used  good  meas- 
ures (i.e.,  low  valid  v.  high  valid),  then  one 
knows  that  a  bias  of  some  sort  is  operating. 

Meta-analysis  uses  systematic  methods  to  un- 
cover trends  in  the  available  research  literature. 
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Assuming  that  there  are  some  good  studies  and 
coding  criteria  can  be  agreed  on,  it  should  prove 
to  be  a  useful  tool  in  understanding  the  research 
on  the  effects  of  psychotherapy.  A  potential 
problem,  however,  is  that  by  focusing  on  avail- 
able research,  one  ignores  the  fact  that  only  re- 
search that  reports  positive  and/or  statistically 
significant  findings  may  be  published.  Although 
some  believe  (241)  that  this  could  be  a  problem 


only  when  there  are  a  few  studies  reporting  sig- 
nificant findings  and  when  the  magnitude  of  the 
differences  between  groups  is  very  small,  obvi- 
ously, the  quality  of  available  research  is  a  criti- 
cal factor,  in  the  usefulness  of  meta-analysis. 
Other  problems,  such  as  the  compatibility  of 
studies,  are  described  in  detail  in  the  next  chap- 
ter as  part  of  a  substantive  review  of  Smith,  et 
al.'s  work. 


FINAL  COMMENTS 

This  chapter  has  described  the  methodologi- 
cal strategies  that  can  be  used  to  assess  psycho- 
therapy. Implicit  in  this  discussion  was  an  as- 
sumption that  psychotherapy  represents  a  re- 
searchable  intervention  that  can  be  evaluated 
using  scientific  criteria  of  measurement  and 
design.  Valid  measures  of  psychotherapy  out- 
come can  be  developed,  and  designs  that  allow 
relatively  unequivocal  assessment  of  psycho- 
therapy can  be  constructed.  Although  there  are 
some  who  maintain  that  it  is  not  possible  to 
assess  psychotherapy  because  of  its  inherent 
complexity,  it  is  not  clear  what  types  of  in- 
formation would  be  excluded  by  the  scientific 
analyses  described  here. 

Whether  the  methods  described  here  have 
been  applied  appropriately  and  to  what  ends 
they  have  been  used,  however,  is  a  different 
question.  Despite  the  possibility  of  conducting 
research  on  the  outcomes  of  psychotherapy,  as 
is  noted  in  the  next  chapter,  well-conducted  re- 


search is  inadequate  to  answer  at  least  some  of 
the  important  questions  about  psychotherapy 
(207,277).  The  reasons  for  the  lack  of  research 
are  not  clear,  but  probably  have  to  do  with  both 
substantive  features  of  psychotherapy  and  a 
complex  set  of  pragmatic  factors. 

These  factors  include  the  difficulty  of  concep- 
tualizing the  multiple  factors  that  are  part  of 
psychotherapy,  as  well  as  attitudinal  differences 
on  the  part  of  researchers  and  therapists.  In 
part,  the  development  of  program  evaluation 
techniques  may  facilitate  the  conduct  of  policy- 
relevant  research.  In  addition,  more  attention, 
by  researchers,  therapists,  and  the  Government 
to  the  issues  of  conducting  ethically  acceptable 
controlled  experiments,  may  help  to  develop 
better  research.  Finally,  attention  to  the  issue  of 
conclusion-making,  based  on  an  analysis  of 
multiple  investigations,  may  lead  to  better  un- 
derstanding of  exactly  what  is  achieved  by  psy- 
chotherapy under  what  conditions. 
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Undoubtedly,  there  is  confusion  about  the  ef- 
fectiveness of  psychotherapy;  both  in  the  pro- 
fessional and  popular  literature,  a  variety  of 
claims  have  been  made  about  the  worth  of  psy- 
chotherapeutic treatments.  Although  it  is  not 
possible  here  to  resolve  fully  the  differences  be- 
tween conflicting  viewpoints,  the  present  chap- 


ter briefly  reviews  the  major  areas  of  dispute 
and  attempts  to  apply  the  methodological  prin- 
ciples described  in  the  previous  chapter  to  assess 
the  various  claims  about  psychotherapy.  The  le- 
gitimate disagreements  among  reviewers  are  ac- 
knowledged, and  an  effort  is  made  to  assess  the 
differences  objectively. 


SCOPE  OF  REVIEW 

Although  no  one  has  classified  all  the  research 
conducted  on  psychotherapy,  it  is  clear  from 
even  a  cursory  review  of  the  literature  that  there 
is  a  plethora  of  research  (see,  e.g.,  208,276).  The 
research  ranges  from  very  specific  analyses 
of  psychotherapeutic  procedures  to  large-scale 
program  evaluations.  Clearly,  there  are  differ- 
ences in  the  usefulness  of  particular  types  of  re- 
search for  the  general  question  of  psychothera- 
py's  efficacy,  but  it  is  difficult  to  differentiate 
research  according  to  a  relevance  criterion. 

While  some  commentators  have  made  a  dis- 
tinction between  basic  and  evaluative  research 
(e.g.,  66,  268),  the  utility  of  any  research  to 
answer  efficacy  questions  is  usually  a  matter  of 
degree.  Basic  research  on  psychotherapy  does 
not  indicate  whether  or  not  treatments  are  effec- 
tive as  practiced,  but  does  provide  some  under- 
standings about  the  nature  of  the  intervention. 
For  those  concerned  with  understanding  the 
processes  underlying  psychotherapy,  such  data 
are  very  important.  Conversely,  evaluative 
studies,  although  they  provide  direct  informa- 
tion about  the  actual  effectiveness  of  a  treat- 
ment, often  yield  equivocal  analyses  of  causali- 
ty and  must  be  interpreted  in  conjunction  with 
basic  research  data.  In  the  present  chapter,  cate- 
gorizations are  avoided;  the  analysis  includes  a 
variety  of  research  studies  which  have  been  con- 
sidered, by  at  least  some  reviewers,  to  be  rele- 


vant to  understanding  the  efficacy  problem. 
Several  additional  caveats  should  also  be  noted. 

One  important  issue  relating  to  how  one  as- 
sesses the  efficacy  literature  is  the  nature  of  the 
problems  for  which  psychotherapy  treatments 
are  offered.  Thus,  a  great  deal  of  research  ex- 
amines the  treatment  of  very  specific  nondis- 
abling  mental /behavioral  problems,  and  it  is 
difficult  to  assess  the  generalizability  of  this 
literature  to  understanding  the  efficacy  of 
treatments  for  very  severe  and  disabling  condi- 
tions. Unfortunately,  the  problem  is  even  more 
complex,  for  much  of  the  psychotherapy  litera- 
ture deals  with  conditions  that  are  difficult  to 
classify.  In  part,  this  problem  has  to  do  with  the 
fact  that  definitions  of  severity  are  relative  and 
dependent  on  the  context  in  which  the  problem 
exists.  The  present  chapter  attempts  to  clarify, 
wherever  possible,  the  degree  to  which  findings 
about  particular  problems  are  generalizable  to 
other  mental  health  problems. 

In  terms  of  the  scope  of  the  present  chapter, 
the  main  focus  is  on  a  description  and  analysis 
of  previous  reviews  of  the  literature.  As  dis- 
cussed in  chapter  3,  no  matter  how  well  con- 
ceived and  executed,  single  studies  have  limited 
use  within  a  scientific  framework.  A  number  of 
very  extensive  reviews  of  the  literature  (i.e., 
discussions  of  multiple  studies)  have  already 
been  conducted,  and  an  effort  is  made  in  the 
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present  chapter  to  describe  these  reviews  as  fully 
as  possible.  Where  appropriate,  as  part  of  this 
analysis,  the  data  and  methods  of  particular 
studies  discussed  by  these  reviews  are  presented.; 
OTA's  decision  to  focus  on  reviews  has  to  do 
both  with  a  goal  to  best  represent  the  available 
literature  and  with  the  fact  that  these  reviews 
comprise  a  focal  point  for  much  of  the  current 
debate  about  psychotherapy's  effectiveness. 

One  omission  from  the  literature  reviews  in 
the  present  chapter  are  popularized  summaries 
of  the  psychotherapy  literature.  For  example, 
Gross'  recent  book  The  Psychological  Society 
has  not  been  included.  Gross'  treatise  is  an  inter- 
esting (although  critical)  report  on  psychothera- 
py, but  is  primarily  a  secondary  source  review 
of  the  literature  (see  also  279,281).  Instead  of 
discussing  such  work  directly,  the  chapter  de- 
scribes the  scholarly  reviews  on  which  Gross 
based  his  book.  Not  included  here  either  are 


some  of  the  scholarly  literature  reviews  which 
have  as  their  primary  focus  the  contrast  of  one 
therapy  or  approach  with  others.  We  were  cen- 
trally interested  in  reviews  that  dealt,  most 
generally,  with  the  problem  of  whether  psycho- 
therapy is  efficacious  under  what  conditions. 
The  central  focus  of  a  review  on  this  question 
guided  OTA's  selections. 

The  reviews  discussed  below  are  presented  in 
approximate  chronological  order  according  to 
their  appearance  in  the  literature.  It  is  hoped 
that  this  ordering  will  give  readers  a  sense  of 
how  research  and  thinking  about  effectiveness 
of  psychotherapy  have  evolved  during  the  past 
several  decades.  The  chapter  also  describes  the 
findings  of  a  number  of  the  most  important 
studies  on  outcome.  These  descriptions  are  pre- 
sented, where  necessary,  to  facilitate  under- 
standing of  the  issues  raised  by  reviewers. 


EFFICACY  REVIEWS 

In  the  reviews  described  below,  the  data  re- 
ported are  reviewed  first;  then,  some  of  the 
commentary  that  has  been  stimulated  by  the  re- 
view is  discussed.  The  methods  used  by  the  re- 
viewer to  select  studies  and  analyze  them  are 
compared  against  the  methodological  criteria 
described  in  chapter  3.  As  noted  above,  the  goal 
is  to  describe  the  research  that  is  most  relevant 
to  the  efficacy  question  and  to  assess  what  ap- 
pear to  be  the  most  reasonable  implications  of 
this  research  literature. 

Eysenck's  Reviews 

The  earliest  and  probably  the  most  controver- 
sial review  of  the  psychotherapy  outcome  litera- 
ture was  conducted  by  Eysenck  almost  30  years 
ago  (70;  see  also  71,72).  Eysenck,  in  his  early 
review,  considered  24  research  articles  which  in- 
cluded 8,053  cases  of  psychotherapy  conducted 
with  neurotic  patients.  He  divided  the  studies, 
on  the  basis  of  his  assessment  of  their  thera- 
peutic approach,  into  two  groups:  1)  psycho- 
analytic therapy  and  2)  eclectic  therapy.  The 
principal  criterion  for  assessing  effectiveness 


was  a  rating  Eysenck  developed  of  improvement 
following  therapy. 

The  most  controversial  aspect  of  Eysenck's  re- 
port was  his  comparison  of  improvement  rates 
in  therapy  against  an  improvement  rate  that  he 
calculated  for  two  no-treatment  groups.  In  the 
eclectic  therapy  studies,  Eysenck  calculated  an 
improvement  rate  for  therapy  of  64  percent 
within  2  years.  He  compared  this  rate  with  what 
was  achieved  in  comparison  groups  of  patients 
who  did  not  have  therapy.  Eysenck  found  that 
the  no-treatment  improvement  rate  was  approx- 
imately the  same  as  that  achieved  in  the  eclectic 
therapy  studies.  The  improvement  rate  calcu- 
lated for  psychoanalysis  was  lower,  approx- 
imately 44  percent,  and  was  below  that  of  the 
no-treatment  groups.  One  source  of  Eysenck's 
data  on  no-therapy  improvement  was  a  study 
by  Denker  (61)  of  patients  with  emotional  prob- 
lems who  were  treated  by  general  medical  prac- 
titioners (another  source  was  Landis  (153)). 
Denker's  data  consisted  of  insurance  company 
records  on  500  individuals  who  had  submitted 
mental  disability  claims.  He  had  found  that 
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within  1  year,  without  receiving  any  specific 
psychotherapy,  44  percent  had  returned  to 
work,  and  that  by  the  end  of  2  years,  an  addi- 
tional 27  percent  had  returned  to  work. 

Eysenck's  work  unleashed  an  extraordinary 
reaction,  some  of  which  still  affects  thinking 
about  psychotherapy  research  (17,18,20,160, 
181,242,268).  Much  of  the  reaction  has  been 
negative,  and,  in  response,  Eysenck  and  his 
colleagues  have  tried  to  refute  claims  of  his 
critics  and  to  provide  additional  data  (e.g., 
70,71).  The  debate  between  Eysenck  and  other 
researchers  is  important  to  unravel,  both  be- 
cause of  its  centrality  in  discussions  about  psy- 
chotherapy, and  because  of  its  implications  for 
the  most  important  questions  about  psychother- 
apy. In  terms  of  Eysenck's  original  report,  three 
types  of  problems  seem  to  be  important  in  inter- 
preting his  analysis:  1)  the  nature  of  the  data 
that  Eysenck  reviewed,  2)  the  utility  of  Denker's 
comparison  data,  and  3)  Eysenck's  interpreta- 
tion of  the  data. 

The  first  problem,  that  of  specificity  of  the 
data,  has  been  raised  by  a  number  of  Eysenck's 
critics  (e.g.,  160).  In  essence,  these  reviewers 
have  claimed  that  it  was  inappropriate  to  gen- 
eralize very  widely  about  psychotherapy  on  the 
basis  of  Eysenck's  data.  The  most  important  dif- 
ficulty is  that  it  is  not  clear  what  was  done  in 
each  of  the  studies.  One  cannot  determine  how 
much  therapy  was  actually  received  by  patients 
and  what  the  quality  of  this  treatment  was  (19). 
It  also  seems  important  to  note  that  the  research 
reported  by  Eysenck  was  conducted  prior  to 
1950,  at  a  time  when  nonpsychodynamically 
based  therapies  were  only  beginning  to  be  used. 
The  generalizability  of  this  work  to  the  types  of 
nonpsychodynamic  therapies  carried  out  today, 
as  well  as  to  currently  practiced  psychoanalytic 
treatments,  is  not  at  all  clear  (see,  e.g.,  269). 

The  second  problem  has  to  do  with  the  "con- 
trol" group  data  used  by  Eysenck  in  his  anal- 
ysis. Eysenck's  control  data  for  the  effects  of 
treatment  were  drawn  from  a  nonrandomly  se- 
lected control  group.  It  is  difficult  to  determine 
how  this  nontreated  group  might  have  differed 
from  individuals  who  received  therapy.  They 
may  have  been  more  or  less  troubled  and  dys- 
functional than  those  who  sought  treatments. 


Not  only  are  the  differences  between  the  control 
and  treatment  group  subjects  not  clear,  but  as 
noted  by  Meltzoff  and  Kornreich  (181),  neither 
Denker's  (61)  nor  Landis'  (153)  data  were  rep- 
resentative (i.e.,  the  data  were  not  derived  from 
a  survey). 

Meltzoff  and  Kornreich  (181)  also  note  that 
the  control  group  data  did  not  really  represent 
no-treatment  data.  Especially  in  Denker's  study, 
where  patients  were  diagnosed  by  general  medi- 
cal practitioners,  a  form  of  treatment  was  pro- 
vided. Patients  were  provided  "sedatives,  ton- 
ics, as  well  as  reassurance,  and  a  placebo  type  of 
treatment."  Furthermore,  since  none  of  these  pa- 
tients was  severely  disturbed  to  begin  with,  the 
return-to-work  measure  may  not  indicate  an  ac- 
tual remission  of  the  symptoms  that  first 
brought  the  individual  to  the  practitioner. 

Finally,  a  number  of  commentators  have 
pointed  to  errors  in  Eysenck's  categorizing  of 
studies  and  errors  in  the  way  he  handled  data. 
Bergin  (19)  reanalyzed  the  data  used  in  Ey- 
senck's original  report  and  finds  a  different  rate 
of  recovery  for  psychoanalytic  treatment.  What 
Bergin  did,  among  other  procedures,  was  to 
eliminate  cases  where  individuals  left  therapy; 
Eysenck  had,  instead,  counted  these  cases  as 
failures.  Bergin  also  claims  to  have  found  cod- 
ing errors  in  Eysenck's  original  report  (e.g.,  out- 
come data  incorrectly  represented).  Bergin's  re- 
analysis  had  yielded  an  improvement  rate  of  65 
percent  for  the  eclectic  therapies  (similar  to 
Eysenck's).  He  notes  that  improvement  rates  for 
all  types  of  treatment  may  be  deceptive,  because 
there  was  a  great  deal  of  systematic  variation 
across  personality  types,  therapists,  and  clinics. 
Bergin  concludes  that  global  questions  of  psy- 
chotherapeutic efficacy  are  "silly"  and  that  one 
must  analyze  specific  therapies  for  specific 
problems. 

Eysenck  acknowledges  many  of  the  methodo- 
logical problems  with  his  data  and  has  argued 
that  the  burden  of  proof  to  provide  evidence 
should  be  on  those  who  seek  to  promote  psy- 
chotherapy (see  223).  He  has  also  updated  his 
original  research,  and,  in  later  reviews  (e.g., 
69),  cites  an  additional  11  studies.  Although  his 
initial  conclusion  about  the  lack  of  effectiveness 
of  psychotherapy  remains,  he  finds  supportive 
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evidence  for  at  least  one  type  of  psychotherapy, 
a  behavioral  approach  based  on  the  work  of 
Wolpe.  It  should  also  be  noted  that  a  number  of 
commentators  (e.g.,  181)  have  criticized  Ey- 
senck  for  selectively  reviewing  the  literature.  By 
the  time  of  his  1965  paper  (which  was  published 
along  with  17  critiques  by  prominent  research- 
ers), there  were  at  least  70  control  group  studies 
of  psychotherapy.  Most  of  these  were  not  in- 
cluded in  Eysenck's  review,  although  his  reasons 
for  not  selecting  these  studies  are  not  clear. 

Although  Eysenck's  work  is  often  interpreted 
as  an  indictment  of  all  psychotherapy,  as  noted 
above,  Eysenck  (69)  cites  evidence  as  to  the 
efficacy  of  therapies  based  on  behavior  theory. 
Thus,  he  describes  a  study  conducted  by  Laza- 
rus (155)  of  phobic  patients  who  were  treated 
either  by  group  desensitization  (a  behavior  ther- 
apy), group  interpretation  (a  psychodynamic 
therapy),  and  a  combination  of  nonbehavioral 
methods.  The  results  showed  a  72-percent  re- 
covery rate  for  the  behavior  therapy  versus  a 
12-percent  recovery  rate  for  the  verbal  therapy. 
Eysenck  also  relies  on  an  unpublished  report  by 
Wolpe  (the  developer  of  desensitization  thera- 
py) in  which  the  results  of  Wolpe's  investiga- 
tions (122  cases)  were  compared  with  reported 
results  from  two  large  psychoanalytic  institu- 
tions (approximately  400  cases).  Wolpe  had  re- 
ported that  90  percent  of  his  cases  were  cured  or 
much  improved  versus  60  percent  of  the  psycho- 
analytically  treated  patients. 

In  trying  to  make  sense  of  the  controversy 
over  Eysenck's  research,  a  number  of  aspects  of 
his  work  should  be  highlighted.  First,  whether 
one  accepts  his  findings  or  not,  the  primary  pur- 
pose of  the  original  review  was  to  assess  the 
data  for  treatment  of  neurotics.  Severe  mental 
disability  was  considered  only  secondarily  in 
later  reports.  Second,  while  Eysenck's  critique 
has  been  interpreted  as  a  critique  of  all  psycho- 
therapy (and  psychotherapy  research),  his  com- 
ments are  most  critical  of  psychoanalytically 
derived  therapies.  He  is  supportive,  as  described 
above,  of  therapies  based  on  behavior  theory. 
Finally,  it  should  be  noted  that  Eysenck  did  not 
find  psychotherapy  to  be  harmful  and,  in  fact, 
found  that  eclectic  therapies  were  associated 
with  fairly  high  rates  of  improvement.  The  criti- 


cal question — whether  improvement  was  caused 
by  psychotherapy  or  resulted  from  other  factors 
(e.g.,  spontaneous  remission,  placebo  effects) 
— is  not  resolved  by  Eysenck's  work. 

Meltzoff  and  Kornreich's  Analysis 

According  to  Meltzoff  and  Kornreich  (181), 
their  work  was  stimulated  by  a  belief  that  there 
had  been  advances  in  the  development  of  re- 
search on  the  effectiveness  of  psychotherapy 
since  Eysenck  (who  initially  published  in  the 
early  1950's),  but  that  this  work  was  underrep- 
resented  in  the  literature.  Meltzoff  and  Korn- 
reich, in  their  book-length  discussion  of  the  psy- 
chotherapy research  literature,  made  an  impor- 
tant departure  from  earlier  reviews  and  clas- 
sified studies  by  methodological  adequacy. 
Groups  of  studies  were  formed  according  to 
their  methodological  quality  and  compared  in 
terms  of  the  positiveness  of  findings.  Meltzoff 
and  Kornreich  wanted  to  come  up  with  a  "fair" 
assessment  of  what  was  contained  in  the  avail- 
able literature. 

In  their  category  of  "adequate"  research 
designs,  they  included  studies  which  used  a 
control  group  condition  and  adequate  outcome 
measures.  In  their  "questionable"  research  de- 
sign category,  they  included  studies  with  con- 
trol groups  that  may  not  have  been  comparable 
and/or  used  poor  outcome  measures  or  analysis 
procedures.  They  also  included  in  this  latter 
category  studies  that  used  analog  designs.  The 
results  of  the  studies  they  assessed  were  catego- 
rized as  either  "positive,"  "negative,"  or  null. 
Since  most  studies  included  multiple  outcome 
measures,  their  subdivision  of  outcomes  was  a 
judgment  of  the  balance  of  statistically  signifi- 
cant findings.  Their  analysis  included  approx- 
imately 100  control  group  studies. 

According  to  Smith,  et  al.  (268),  who  tabu- 
lated the  results  of  Meltzoff  and  Kornreich's  re- 
view, 80  percent  of  the  controlled  studies  they 
reported  yielded  positive  results,  while  the  other 
20  percent  had  null  or  negative  results.  There 
was  also  a  positive  relationship  between  re- 
search quality  and  positive  findings.  Thus,  84 
percent  of  Meltzoff  and  Kornreich's  adequately 
designed  studies  yielded  positive  results  about 
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the  effectiveness  of  psychotherapy,  while  only 
33  percent  of  the  studies  of  questionable  design 
yielded  positive  results. 

Meltzoff  and  Kornreich  (181)  also  assessed 
the  degree  to  which  results  were  "major"  or  "mi- 
nor." The  studies  categorized  as  "adequately  de- 
signed, with  major  outcomes"  varied  on  a  num- 
ber of  dimensions.  These  studies  included  re- 
search such  as  Grace,  Pinski,  and  Wolff's  (107) 
assessment  of  group  therapy  for  individuals 
with  ulcerative  colitis.  In  this  study,  patients  in 
the  treatment  group  received  psychotherapy  to 
alleviate  stress.  Over  a  2-year  period,  it  was 
found  across  a  variety  of  "hard"  criteria  (e.g., 
operations  required)  that  those  who  received 
group  therapy  were  less  likely  to  require  medi- 
cal treatment  than  those  who  did  not  (there  was 
a  lower  morbidity  rate  for  therapy  patients,  as 
well). 

A  different  type  of  study  included  by  Meltzoff 
and  Kornreich  (181)  was  Morton's  (190)  study 
of  college  students  who  were  referred  to  psycho- 
therapy for  severe  personal  problems.  A  ran- 
domized control  group  design  was  implemented 
by  having  some  subjects  placed  on  a  waiting 
list.  Raters,  who  were  blind  to  information 
about  the  students'  treatment  (they  were  the 
counselors  who  made  the  actual  referral),  talked 
with  students  after  3  months.  On  a  number  of 
measures,  including  those  that  assessed  cogni- 
tive and  behavioral  variables,  93  percent  of  the 
treatment  group  students  and  only  47  percent  of 
the  control  group  students  showed  improve- 
ment. The  students  who  received  psychotherapy 
were  considered  to  have  received  a  major 
benefit. 

Although  it  is  difficult  to  systematically 
analyze  the  types  of  studies  included  in  Meltzoff 
and  Kornreich's  (181)  review,  most  of  the  stud- 
ies they  report  appear  not  to  be  of  severely  dis- 
abled patients.  In  fact,  almost  all  the  studies  in 
which  there  were  adequate  designs  and  major 
positive  benefits  were  studies  in  which  the  prob- 
lems were  relatively  specific  and  the  patients 
were  not  institutionalized.  The  reason  for  this 
may  be  connected  with  the  problems  of  doing 
control  group  research,  as  well  as  with  the  na- 
ture of  psychotherapy.  As  noted  in  chapter  3,  it 
is  easier  to  conduct  well-designed  control  group 
research  when  problems  are  specific  and  the 


potential  harm  of  withholding  treatment  is  not 
severe.  The  hypothesis  must  be  left  open  that 
the  more  general  and  unspecified  the  problem, 
the  less  efficacious  is  psychotherapy. 

This  point  has  been  made  by  Malan  (168) 
about  Meltzoff  and  Kornreich's  work  (181). 
Malan  noted  that,  although  he  agreed  with  the 
Meltzoff  and  Kornreich  conclusion  about  the 
beneficial  effects  of  psychotherapy,  its  general- 
izability  for  a  variety  of  patient  groups  was  not 
clear.  There  is  also  a  problem,  in  generalizing 
from  Meltzoff  and  Kornreich,  of  not  knowing 
what  specific  techniques  work  with  what  specif- 
ic problems.  Although  their  methodological  fo- 
cus is  useful  for  assessing  hypotheses  about  the 
relation  between  research  quality  and  outcome, 
it  obscures  the  synthesis  of  findings  about 
specific  treatments.  From  a  more  optimistic  per- 
spective, it  is  comforting  that  their  positive  find- 
ings rate  is  a  bit  higher  than  that  of  Eysenck, 
and  further,  that  these  positive  results  were  ob- 
tained in  well-designed  studies. 

Bergin's  Reviews 

Bergin's  1971  review  (19)  of  psychotherapeu- 
tic effectiveness  was  described  briefly  above  in 
reference  to  his  critique  of  Eysenck's  work. 
Bergin's  analysis  is  important  (see  also  18,21) 
because,  unlike  earlier  critics  of  Eysenck,  he  ac- 
tually examined  the  studies  used  in  the  original 
report  (72)  and  recomputed  Eysenck's  treatment 
remission  rates.  Bergin  used  different  assump- 
tions and  procedures  for  establishing  improve- 
ment rates,  and,  as  noted  above,  he  found  a  dis- 
crepancy in  Eysenck's  analysis  of  psychody- 
namic  therapy.  In  addition,  Bergin  (19)  calcu- 
lated a  different  remission  rate  for  no-treatment 
(based  on  data  not  discussed  by  Eysenck),  and 
found  that  only  about  30  percent  of  the  patients 
would  have  recovered  had  there  not  been  psy- 
chotherapy. Within  the  framework  of  Eysenck's 
nonexperimental  data  base,  there  are  limits  to 
one's  certainty  about  any  conclusions. 

Bergin  (19),  in  part  to  remedy  this  problem, 
reviewed  52  studies  of  psychotherapy  outcome. 
He  claimed  that  these  studies  represented  a 
"cross-section"  of  the  available  literature  (see 
276).  Approximately  half  of  the  studies  had  con- 
trol groups,  and  Bergin  categorized  the  studies 
on  the  basis  of  a  number  of  variables,  including 
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the  adequacy  of  the  research  design,  duration  of 
therapy,  type  of  therapy,  and  therapist  experi- 
ence. Studies  were  judged  and  cross-tabulated  in 
terms  of  the  results  of  the  therapy:  "positive," 
"negative,"  or  "in  doubt." 

Of  the  52  studies,  Bergin  (19)  judged  22  to  be 
positive  (indicating  psychotherapy  was  effec- 
tive), he  judged  15  as  having  negative  results, 
and  15  were  judged  "in  doubt."  Bergin  con- 
cluded that  "psychotherapy,  on  the  average,  has 
moderate  positive  results."  Typical  of  the  stud- 
ies Bergin  classified  as  positive  was  Gottschalk, 
Mayerson,  and  Gottlieb's  (106)  study  of  brief 
therapy.  Using  a  no-treatment  control  group  de- 
sign, Gottschalk,  et  al.  studied  neurotics  who 
were  given  emergency  outpatient  therapy.  The 
treated  patients  improved  significantly  more 
than  the  controls  on  a  variety  of  behavioral  and 
cognitive  measures.  Another  positive  study 
cited  by  Bergin  was  Vorster's  (288)  investigation 
of  neurotics  treated  by  eclectic  therapies. 
Vorster  found  the  treatment  group  significantly 
improved,  although  this  study's  design  did  not 
employ  a  randomized  control  group. 

In  a  more  recent  extension  of  Bergin's  (19) 
review,  Bergin  and  Lambert  (21)  discuss  addi- 
tional evidence  to  support  their  hypothesis  that 
psychotherapy  is  efficacious.  They  also  reply  to 
the  criticism  (e.g.,  222)  of  Bergin's  original 
work.  Bergin  and  Lambert  cite  as  one  of  the  best 
examples  of  more  recent  research  a  study  by 
Sloane,  Staple  ,  Christol,  Yorkston,  and  Whip- 
ple (266).  That  study  used  90  outpatients,  of 
whom  two-thirds  were  neurotic  and  one-third 
had  personality  conduct  disorders.  The  patients 
were  randomly  assigned  (they  were  also 
matched  with  respect  to  sex  and  severity  of 
symptoms)  to  short-term  analytically  oriented 
therapy,  behavior  therapy,  or  a  minimal  treat- 
ment wait-list  group.  Treatment  outcome  was 
assessed  after  4  months.  On  average,  all  groups 
improved  significantly  across  a  set  of  target 
symptoms,  but  the  two  therapy  groups  (analyti- 
cally-oriented and  behavior)  improved  signifi- 
cantly more  than  the  minimal  treatment  wait- 
list group.  No  significant  differences  were  found 
in  outcomes  between  the  actual  therapy  groups. 

Bergin  and  Lambert  (21)  conclude  that,  "Psy- 
choanalytic/insight therapies,  humanistic  or  pa- 


tient-centered psychotherapy,  many  behavioral 
techniques  and,  to  a  lesser  degree,  cognitive 
therapies  rest  on  a  reasonable  empirical  base." 
They  contend  that  these  therapies  achieve  re- 
sults that  are  superior  to  no-treatment  and  to 
various  placebo  treatment  procedures.  Bergin 
and  Lambert  also  assert  that  even  if  one  accepts 
Eysenck's  "two-thirds  remission  within  2  years" 
formula,  there  is  still  positive  evidence  for  psy- 
chotherapy. This  is  because  treatment  effects  of 
the  same  magnitude  are  frequently  obtained  in  6 
months  or  less  in  formal  psychotherapy,  "...  a 
considerable  evidence  of  therapy's  efficiency/ 
efficacy  over  no  treatment."  In  effect,  psycho- 
therapy hastens  the  recovery  period  and  reduces 
both  suffering  and  the  effects  of  disability. 

Rachman's  Critique 

As  can  be  seen  from  the  above  two  reviews, 
the  early  1970's  was  a  fertile  time  for  psycho- 
therapy assessments.  Rachman  (222),  a  frequent 
collaborator  of  Eysenck's,  provided  another  re- 
view which  served,  in  part,  to  refute  Eysenck's 
critics.  Rachman,  in  particular,  responded  to 
Bergin's  (19)  reanalysis  of  Eysenck's  data  and  to 
the  dispute  about  what  studies  should  be  in- 
cluded in  a  review  of  psychotherapy  efficacy. 
Rachman,  in  analyzing  Bergin's  work,  dis- 
allowed a  number  of  studies  (5  out  of  14),  be- 
cause the  subjects  were  delinquents  or  had  psy- 
chosomatic complaints,  rather  than  being  neu- 
rotic. If  Rachman's  analysis  had  been  extended 
to  Meltzoff  and  Kornreich's  (181)  review,  the 
same  problem  would  have  existed. 

Rachman  (222)  analyzed  23  studies  which  he 
considered  to  be  relevant  for  assessing  the  ef- 
ficacy of  "verbal"  psychotherapy  (i.e.,  therapies 
based  on  conversation  between  therapist  and 
patient — primarily,  humanistic  and  psychody- 
namically  based).  Of  these  studies,  Rachman 
found  that  only  one  provided  tentative  evidence 
of  the  effectiveness  of  psychotherapy.  Five  stud- 
ies produced  negative  effects,  where  treatment 
group  results  failed  to  exceed  control  groups  or 
base-line  remission  rates.  It  is  important  to  note 
that  Rachman  excluded  17  studies  for  a  variety 
of  reasons,  only  2  of  which  showed  negative  ef- 
fects. The  reasons  for  excluding  the  15  studies 
that  showed  positive  results  varied,  and  in- 
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eluded  the  use  of  unacceptable  outcome  meas- 
ures (e.g.,  a  projective  test),  the  exclusion  of 
subjects  who  left  therapy  before  treatment  was 
completed  (which  Rachman  said  inflated  im- 
provement rates),  and  inconsistency  across  out- 
come measures  (i.e.,  some  measures  showed 
positive  effects,  while  one  or  more  measures 
showed  negative  effects).  The  two  negative 
studies  were  excluded  because  randomized  con- 
trol group  conditions  were  not  employed. 

A  number  of  commentators  (e.g.,  257,268) 
have  reviewed  Rachman's  (222)  critique  and 
analysis,  and  they  claim  that  he  used  inconsist- 
ent standards  for  evaluating  evidence.  In  partic- 
ular, they  assert  that  Rachman  used  criteria  for 
assessing  the  verbal  psychotherapies  that  were 
different  from  the  criteria  he  used  for  the  behav- 
ioral therapy  experiments.  Smith,  et  al.  (268) 
have  criticized  Rachman  because  he  selectively 
chose  studies  to  review.  They  indicate  that 
many  more  studies  were  available  to  Rachman 
than  he  included,  and  add  that  it  cannot  be  in- 
ferred from  his  discussion  why  particular  stud- 
ies were  not  referenced.  Smith,  et  al.  also  criti- 
cize Rachman  for  "ex  post  facto"  exclusion  of 
studies  based  on  methodological  criteria.  They 
argue  that  he  should  have  compared  the  good 
and  poor  designs  to  determine  whether  or  not 
they  yielded  different  kinds  of  conclusions,  as 
Meltzoff  and  Kornreich  (181)  did  in  their 
analysis. 

Luborsky,  Singer,  and 
Luborsky's  Review 

In  a  more  current  assessment,  Luborsky, 
Singer,  and  Luborsky  (162)  tried  to  evaluate  all 
reasonably  controlled  studies  of  psychotherapy 
on  "real"  patients  (i.e.,  analog  studies  were  ex- 
cluded). Luborsky,  et  al.  examined  available 
studies  that  had  assessed  therapy  for  treating 
recognized  problems  of  individuals  who  sought 
psychological  treatment.  Many  of  these  studies 
were  ones  that  had  been  used  by  Bergin  (19)  and 
by  Meltzoff  and  Kornreich  (181).  Although  Lu- 
borsky, et  al.'s  definition  excluded  some  behav- 
ioral research  (primarily  because  it  was  not  con- 
ducted in  actual  clinical  settings)  and  some  pa- 
tient populations,  their  scope  was  wider  than 
that  of  other  reviewers. 


Each  of  the  studies  Luborsky,  et  al.  (162)  as- 
sessed was  categorized  on  a  number  of  dimen- 
sions and  then  summarized  in  a  "box-score" 
analysis  (see  ch.  3).  One  central  dimension  was 
research  quality,  which  was  determined  on  the 
basis  of  12  criteria.  The  criteria  included  the 
study's  method  for  assigning  subjects  to  com- 
parison groups,  the  procedures  for  dealing  with 
premature  therapy  termination,  experience  of 
therapists,  tailoring  of  outcome  measures  to 
therapeutic  goals,  and  the  adequacy  of  the  sam- 
ple size.  Luborsky,  et  al.  summarized  the  cod- 
ings by  "grading"  each  study  on  a  5-point  scale 
of  research  quality.  They  then  categorized  re- 
sults in  terms  of  whether  there  were  significant 
differences  showing  better  effects  for  the  treat- 
ment group  (  +  ),  the  comparison  group  (  —  ),  or 
no  significance  between  the  groups  (0). 

Luborsky,  et  al.  (162)  found  33  studies  in 
which  psychotherapy  treatment  groups  were 
compared  with  no-treatment  control  groups.  Of 
these,  20  studies  yielded  psychotherapy  treat- 
ment groups  which  were  significantly  better  off 
than  control  groups,  and  13  showed  no  differ- 
ence. Luborsky,  et  al.  did  not  find  any  instance 
of  a  control  group  being  better  than  a  psycho- 
therapy treatment  group.  They  found  19  studies 
in  which  schizophrenic  populations  (i.e.,  severe- 
ly disabled  individuals)  were  studied.  Of  these, 
11  yielded  results  in  favor  of  the  psychotherapy 
condition,  and  8  yielded  no  differences.  Lubor- 
sky, et  al.  also  found  that  in  a  majority  of  com- 
parisons (13  out  of  19),  there  were  no  significant 
differences  in  outcomes  to  patients  between  be- 
havior therapy  and  other  psychotherapies. 
While  the  reviewers  note  the  positive  research 
findings  on  behavior  therapies,  they  suggest  the 
need  for  more  studies  in  which  behavior  ther- 
apies are  applied  to  patients  who  have  general- 
ized maladjustments. 

Luborsky,  et  al.  (162)  conclude  that  control 
studies  find  that  patients  who  go  through  psy- 
chotherapy do,  in  fact,  gain.  Because  they 
used  "box  scores"  where  effect  sizes  were  not 
estimated  (see  221),  it  is  not  possible  to  deter- 
mine how  strong  these  effects  will  be.  However, 
according  to  Smith,  et  al.  (268),  there  has  been 
no  published  substantive  criticism  of  Luborsky, 
et  al.  and  (as  is  described  below)  Luborsky,  et 
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al.'s  general  finding  has  been  substantiated  by 
other  reviews. 

NIMH  Synthesis 

Perhaps  the  most  comprehensive  review  of 
psychotherapy  outcome  research  has  been  con- 
ducted at  the  National  Institute  of  Mental 
Health  by  Parloff  and  his  colleagues  (208).  This 
review  provided  an  assessment  of  psychosocial 
treatments  for  mental  disorders  and  was  pre- 
pared for  the  Institute  of  Medicine  (National 
Academy  of  Sciences)  as  part  of  that  Institute's 
work  for  the  President's  Commission  on  Mental 
Health  (219).  The  Parloff,  et  al.  review  differs 
from  earlier  works  in  that  it  uses  a  more  nar- 
rowly defined  treatment  and  is  organized  pri- 
marily according  to  disabling  conditions.  Thus, 
for  each  of  a  variety  of  mental  disorders,  the 
available  evidence  as  to  the  effectiveness  of  psy- 
chotherapy was  assembled  and  analyzed.  The 
reviewers  tried  to  make  some  general  statements 
about  psychosocial  therapies  (psychotherapies 
that  do  not  use  drug  treatments)  and  how  these 
therapies  are  affected  by  other  variables,  such  as 
therapist  and  patient  characteristics.  Parloff,  et 
al.'s  work  is  very  extensive,  so  a  detailed  sum- 
mary is  not  attempted  here;  only  the  work's  gen- 
eral conclusions  are  described  below. 

Parloff,  et  al.'s  (208)  general  finding  (see  also 
207)  was  that  "patients  treated  by  psychosocial 
therapies  show  significantly  more  improvement 
in  thought,  mood,  personality,  and  behavior 
than  do  comparable  samples  of  untreated  pa- 
tients." These  reviewers  found  that  spontaneous 
remission  rates  developed  from  separate  sam- 
ples provide  evidence  that  psychosocial  treat- 
ments seem  to  result  in  greater  improvement 
than  would  be  expected  without  psychothera- 
peutic treatment.  Their  finding  is  supported 
most  clearly  for  disorders  such  as  "anxiety 
states,  fears,  and  phobias."  Parloff,  et  al.'s 
relatively  positive  assessment,  however,  was  ac- 
companied by  a  number  of  caveats.  For  many 
disorders,  psychotherapy  alone  (i.e.,  without 
other  treatments  such  as  drugs)  has  not  been 
demonstrated  to  be  effective;  nor  does  it  appear 
to  be  effective  for  particular  populations  (e.g., 
children).  Finally,  the  review  notes,  although  ef- 
fectiveness evidence  exists  for  a  number  of  dis- 


orders, only  some  types  of  therapies  (with  par- 
ticular therapists)  may  be  effective. 

The  central  aspect  of  Parloff,  et  al.'s  (208) 
review  was  a  summary,  by  each  psychopatho- 
logical  condition,  of  the  available  treatment 
research  evidence.  To  appreciate  the  complexity 
of  this  task,  consider  their  discussion  of  severe 
mental  disorders  such  as  schizophrenia.  For 
these  disorders,  Parloff,  et  al.  found  that  indi- 
vidual and  group  psychotherapy  provide  an 
ambiguous  amount  of  improvement  for  institu- 
tionalized patients;  however,  in  conjunction 
with  drug  therapies  and  other  psychological 
treatments,  they  appear  to  have  important  ef- 
fects. The  authors  note  that  ".  .  .  drugs  do  not 
teach  individual  social  and  interpersonal  skills." 
For  such  hospitalized  populations,  however, 
Parloff,  et  al.  found  considerable  evidence  that 
a  specific  type  of  therapy  (behavior  based)  im- 
proves social  adjustment  (on  a  variety  of  social 
and  interpersonal  variables).  They  also  found 
that  the  return  of  severely  disturbed  patients  to 
their  community  had  positive  effects  on  treat- 
ment outcome,  although  this  finding  was  lim- 
ited to  patients  with  certain  interaction  skills, 
and  under  the  condition  that  the  patient  returns 
to  a  "good"  family  situation. 

One  important  feature  of  the  Parloff,  et  al. 
work,  both  as  it  was  presented  in  the  original 
form  (208)  and  as  it  was  summarized  by  the 
President's  Commission  on  Mental  Health 
(219),  is  that  this  report  examines  the  evidence 
for  alternative  hypotheses  about  the  effects  of 
psychotherapy.  In  particular,  Parloff,  et  al. 
found  a  variety  of  reported  spontaneous  remis- 
sion rates;  that  is,  different  improvement  rates 
are  obtained  for  disturbed  patients  who  do  not 
receive  therapy  (see,  e.g.,  280).  Despite  the  fact 
that  patients  improve  to  some  extent  without 
therapy,  however,  the  hypothesis  that  such 
spontaneous  remission  effects  account  for 
changes  in  treated  patients  cannot  be  validated. 
Parloff,  et  al.  (208)  report  that  studies  which 
have  been  controlled  for  placebo  effects  find 
that  changes  associated  with  treatment  are 
greater  than  those  associated  with  the  placebo. 
Unfortunately,  there  is  an  inherent  problem  in 
theoretically  identifying  the  nature  of  a  psycho- 
therapy placebo  (see  ch.  3)  and  separating  it 
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from  the  effects  of  the  treatment,  so  these  results 
must  be  regarded  as  tentative. 

Parloff  (207),  in  a  discussion  of  the  review 
(208),  calls  for  the  development  of  clinical  trial 
research  to  assess  more  widely  and  systematical- 
ly the  effects  of  psychotherapy.  While  Parloff 
adopts  a  relatively  positive  view  of  the  effects  of 
psychotherapy,  he  finds  the  current  research  lit- 
erature limited.  He  proposes  the  use  of  exper- 
imental designs  to  assess  the  specific  conditions 
(including  aspects  of  the  treatment,  therapist, 
and  patient)  under  which  therapy  procedures 
yield  particular  outcomes.  Although  one  view 
of  such  clinical  trials  would  be  to  test  different 
theories  of  psychotherapy,  an  implication  of 
Parloff,  et  al.'s  work  is  that  other  factors  need  to 
be  incorporated  in  the  design  of  these  trials. 

One  additional  note  about  Parloff,  et  al.'s 
(208)  findings  has  to  do  with  their  review  of  be- 
havior-based therapies.  Parloff,  et  al.,  along 
with  Eysenck,  Rachman,  Bergin,  and  to  some 
extent,  Meltzoff  and  Kornreich,  report  clear-cut 
evidence  that  behavior-based  therapies  are  ef- 
fective treatments  for  specified  conditions. 
Although  its  generalizability  has  not  been 
established  across  the  range  of  disorders,  this 
finding  suggests  one  important  focus  for  future 
research. 

Smith,  Glass,  and  Miller's 
Meta-Analysis 

The  final  review  considered  here,  that  by 
Smith,  Glass,  and  Miller  (268),  has  the  potential 
to  be  as  controversial  as  Eysenck's  original 
work,  and  may  stimulate  an  entire  new  set  of 
psychotherapy  outcome  analyses.  The  meta- 
analytic  methods  used  by  Smith,  et  al.  (see  ch. 
3)  are  at  the  heart  of  the  controversy  about  this 
work.  A  preliminary  published  report  of  the 
meta-analysis  of  375  control  group  studies  of 
psychotherapy  (267)  has  already  stimulated  a 
variety  of  critiques  (e.g.,  73,90,218).  Smith, 
al.'s  goal  was  to  determine  the  state  of  knowl- 
edge about  the  effects  of  psychotherapy,  using 
systematic  scientific  procedures.  Their  meta- 
analytic  procedure  was  used  to  integrate  the 
findings  of  a  disparate  set  of  studies  on  psycho- 
therapy. In  addition,  each  of  the  studies  in  the 
sample  was  classified  to  enable  determination  of 
the  factors  that  influence  outcome  findings. 


In  order  to  conduct  their  review,  Smith,  et  al. 
(268)  tried  to  include  all  controlled  studies  of  the 
effectiveness  of  any  form  of  psychotherapy. 
Controlled  studies  were  defined  as  investiga- 
tions where  a  treatment  group  received  psycho- 
therapy, and  another  group  which  was  compa- 
rable did  not  receive  treatment.  In  some  cases, 
receiving  treatment  meant  a  placebo  or  alternate 
form  of  psychotherapy.  Smith,  et  al.  considered 
a  study  to  be  relevant  if  the  study  investigated 
therapy  that  involved:  1)  patients  who  were 
identified  as  having  an  emotional  or  behavioral 
problem,  2)  treatment  that  was  psychological  or 
behavioral,  and  3)  therapists  who  were  identi- 
fied professionals.  The  definition  resulted  in  the 
analysis'  including  a  variety  of  investigations  of 
treatments  applied  to  problems  of  different  de- 
grees of  severity.  Smith,  et  al.  excluded  some 
types  of  treatment,  including  those  in  which 
psychoactive  drugs  were  used  (these  studies 
were  analyzed  separately),  those  which  were 
primarily  educational,  and  those  which  were 
not,  essentially,  psychosocial  treatments.  They 
surveyed  a  great  number  of  sources  to  identify 
studies,  including  published  journals,  disserta- 
tions, and  clearinghouses  that  identify  profes- 
sional publications.  Their  final  sample  included 
475  controlled  studies  of  psychotherapy. 

Probably  the  most  important  aspect  of  Smith, 
et  al.'s  (268)  analysis  was  the  way  they  classified 
the  research  studies.  Each  study  was  coded  on  a 
number  of  dimensions  related  to  the  character- 
istics of  the  researcher,  therapist,  and  the  pa- 
tient (including  diagnosis);  most  importantly, 
the  studies  were  classified  on  a  series  of  meth- 
odological criteria.  The  methodological  cate- 
gories included  the  nature  of  the  assignment  to 
conditions  (e.g.,  random  v.  matching)  and  such 
factors  as  experimental  mortality  and  internal 
validity  (see  41,165).  Smith,  et  al.'s  principal 
dependent  measure  was  a  standardized  score  for 
the  size  of  the  effect.  From  the  data  reported  in 
each  study,  Smith,  et  al.  calculated  scores  for 
the  size  of  the  effect;  to  allow  comparison  across 
studies,  they  computed  each  as  a  standard 
score.  Studies  often  included  more  than  one  out- 
come measure,  so  Smith,  et  al.'s  analysis  treated 
each  variable  as  a  separate  case.  Thus,  from  the 
475  studies  Smith,  et  al.  analyzed,  they  found 
1,766  effect  size  measures.  In  addition,  they 
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coded  outcome  measures  in  terms  of  the  type  of 
measure,  instrument,  and  its  reactivity  (i.e., 
susceptibility  to  social  desirability,  faking,  etc.). 

Smith,  et  al.'s  (268)  principal  finding  was 
that,  on  the  average,  the  difference  between 
average  scores  in  groups  receiving  psychothera- 
py and  untreated  control  groups  was  0.85  stand- 
ard deviation  units  (i.e.,  the  effect  size  difference 
was  0.85).  According  to  Smith,  et  al.,  this  aver- 
age effect  size  can  be  translated  to  indicate  that 
the  average  person  who  receives  therapy  is  bet- 
ter off  than  80  percent  of  the  persons  who  do 
not.  They  found  little  evidence  for  the  existence 
of  harmful  effects  of  psychotherapy  (i.e.,  very 
few  cases  where  the  mean  of  the  control  group 
was  higher  than  the  treatment  group).  Smith,  et 
al.  found  some  significant  differences  across  the 
types  of  therapies  whose  effects  were  studied 
(the  range  was  0.14  to  2.38),  but  these  effects  are 
confounded  by  variables  such  as  patient  and 
therapist  characteristics  which  were  distributed 
unequally  among  the  therapies.  Finally,  their 
methodological  categories  proved  not  to  corre- 
late with  effect  sizes;  thus,  for  example,  the  bet- 
ter designed  studies  did  not  yield  less  positive 
findings. 

OTA  has  had  an  opportunity  to  review 
Smith,  et  al.'s  (268)  coded  data  of  the  475  stud- 
ies. A  sample  of  studies  was  drawn,  and  their 
reliability  was  checked  by  comparing  our  blind 
codings  with  their  original  data.  In  particular, 
effect  size  ratings  for  a  dozen  studies  were 
recomputed.  This  analysis  indicated  that  the 
Smith,  et  al.  codings  were  both  easily  replicable 
and  apparently  reliable.  Their  validity  is  more 
difficult  to  establish. 

A  key  question  is  whether  the  effect  size 
scores  calculated  by  Smith,  et  al.  (268)  are  a 
valid  measure.  There  are  reasons  to  suggest  that 
their  average  effect  size  is  inflated,  but  there  are 
also  reasons  to  suggest  that  it  is  conservative. 
Inflation  in  the  effect  size  measure  may  have 
come  about  because  only  well-designed  studies 
were  included  in  Smith,  et  al.'s  analysis.  Thus, 
to  the  extent  that  the  control  group  studies  are 
completed  only  for  successful  psychotherapies 
or  for  a  limited  range  of  psychotherapy  treat- 
ments, the  sample  may  be  biased  positively 
when  compared  to  psychotherapy  research.  It  is 


difficult  to  determine  the  exact  nature  of  the  bias 
that  may  result  from  this  problem.  There  are 
also  some  aspects  of  Smith,  et  al.'s  procedures 
which  suggest  that  their  average  effect  size 
measures  may  be  conservative.  Smith,  et  al. 
considered  placebo  treatments,  as  well  as  actual 
forms  of  therapy  and  counseling,  in  their  treat- 
ment group  means.  These  are  all  not  legitimate 
therapies,  and,  in  fact,  when  separately  ana- 
lyzed, showed  lower  effectiveness  as  compared 
to  actual  psychotherapy. 

Most  critics  of  Smith,  et  al.  (268),  in  par- 
ticular Eysenck  (73),  have  disputed  the  meta- 
analysis approach  (actually,  the  published  cri- 
tiques refer  to  Smith  and  Glass  (267)),  because  it 
lumps  together  too  many  things  and  includes 
studies  of  poor  design,  as  well  as  good  design. 
This  criticism  is  potentially  justifiable,  but  a 
close  review  of  Smith,  et  al.  indicates  that  they 
control  for  this  problem  by  their  classification 
of  studies  according  to  methodological  criteria. 
Not  only  do  they  start  with  a  group  of  relatively 
well-designed  studies  (in  terms  of  their  defini- 
tion of  controlled  psychotherapy  research),  but 
they  provide  analyses  of  the  relationship  be- 
tween effect  size  and  classification  variables 
such  as  internal  validity  (a  measure  of  the  quali- 
ty of  the  research  design — see  41).  The  correla- 
tions are  close  to  zero,  indicating  that  studies 
that  use  randomized  control  group  designs  find 
the  same  effect  as  studies  that  use  poorer 
designs.  There  may  still  be  a  sampling  problem 
in  that  the  published  literature  only  reports 
well-controlled  studies,  but  this  problem  is  dif- 
ferent from  that  on  which  most  of  the  criticism 
of  Smith,  et  al.'s  meta-analytic  procedures  is 
based. 

At  this  point,  it  is  difficult  to  know  how  to 
utilize  the  results  of  Smith,  et  al.  (268).  Their 
work  is  certainly  more  systematic  than  that  of 
other  researchers;  however,  their  methods  are 
not  yet  widely  accepted  and  their  work  has  not 
been  available  long  enough  for  comprehensive 
reviews  to  appear.  Perhaps,  the  most  important 
limitation  of  such  meta-analytic  research  is  that 
it  relies  on  the  existing  literature.  It  seems  clear 
that  much  better  research  on  psychotherapy  can 
be  done,  and  a  more  definitive  meta-analysis 
may  have  to  await  the  completion  of  this  new 
research. 
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DISCUSSION 

If  one  considers  only  the  trend  of  findings 
reported  by  scholarly  reviews  and  analyses  of 
the  psychotherapy  outcome  literature,  it  would 
appear  that  psychotherapy  treatments,  under 
some  conditions,  have  been  shown  to  be  effica- 
cious (see,  e.g.,  287).  Although  the  evidence  is 
not  entirely  convincing,  the  currently  available 
literature  contains  a  number  of  good-quality 
research  studies  which  find  positive  outcomes 
for  psychotherapy.  There  are  also  a  large 
number  of  studies  which  report  positive  effects, 
but  whose  methods  or  generalizability  are  dif- 
ficult to  assess.  The  quality  of  the  evidence 
varies  in  terms  of  the  nature  of  the  treatment 
and  the  patient's  problem.  One  difficulty  with 
the  available  literature  is  that,  while  a  host  of 
factors  have  been  identified  as  important  to  psy- 
chotherapeutic outcomes,  the  role  of  these  fac- 
tors (e.g.,  characteristics  of  the  patient,  thera- 
pist, setting)  has  not  been  assessed  in  any  defin- 
itive way. 

Frank  (85),  for  example,  in  commenting  on 
the  state  of  psychotherapy  outcome  research, 
reports  a  disappointment  that  research  has  not 
produced  more  specific  understandings  of  what 
occurs  during  therapy.  While  he  supports  the 
conclusion  that  research  demonstrates  that  psy- 
chotherapy is  somewhat  better  than  no  therapy, 
he  also  finds  that  much  of  the  literature  in- 
dicates equivocal  outcomes.  One  reason  for 
these  equivocal  findings,  according  to  Frank,  is 
that  the  personal  qualities  of  the  therapist  and 
patient  and  their  interaction  may  be  more  im- 
portant than  the  therapeutic  method.  Because 
these  factors  vary  so  widely  and  are  not  typical- 
ly  controlled   for,   variability   is   built  into 


psychotherapy  studies,  making  it  more  difficult 
to  detect  significant  treatment  effects.  Frank 
urges  more  research  that  clearly  assesses  these 
components  of  therapeutic  treatments. 

It  is  likely,  given  the  theoretical  potential  to 
conduct  such  research  on  the  ingredients  of 
psychotherapy,  that  a  more  extensive  research 
literature  could  be  developed  (see  also  207).  It 
probably  means  that  psychotherapy  evalua- 
tions, unlike  evaluations  of  new  drugs,  need  to 
pay  equal  attention  to  the  conditions  of  treat- 
ment and  the  treatment  per  se.  Research  done  on 
psychotherapy  in  settings  different  from  actual 
practice  will  probably  have  limited  utility  and 
will  represent  only  one  stage  of  a  testing  pro- 
gram. Although  it  would  be  hoped  that  psycho- 
therapy research  could  lead  to  better  under- 
standings of  the  role  of  therapist  characteristics 
and  skills,  it  might  only  be  possible  to  assess 
how  these  factors  interact  with  treatments  as 
they  are  actually  delivered.  Finally,  it  should 
also  be  noted  that  some  forms  of  potential  out- 
come research,  such  as  program  evaluations,  do 
not  seem  to  have  made  a  substantial  impact  on 
the  literature  (cf.  181,212).  While  program  eval- 
uations have  been  done  of  a  variety  of  process 
factors  in  mental  health  delivery  systems,  these 
studies  have  less  frequently  measured  outcomes 
using  experimental  designs.  In  part,  this  reflects 
the  developing  nature  of  the  methods  for  pro- 
gram evaluation  (see  ch.  3)  and  the  slow  shift  to 
the  funding  of  such  research.  It  is  possible  that 
this  situation  will  change  with  increased  pres- 
sures for  accountability  and  with  increased  em- 
phasis on  understanding  complex  sets  of  psy- 
chotherapy technologies. 
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The  preceding  chapters  illustrate  some  of  the 
complexity  of  assessing  the  outcomes  of  psycho- 
therapy. Although  such  assessment  is  undoubt- 
edly difficult,  it  nevertheless  seems  possible  to 
evaluate  psychotherapy  using  scientific  methods 
of  analysis.  Assessing  the  costs  and  benefits  of 
psychotherapy,  and  developing  comparisons 
among  effects,  costs,  and  benefits — that  is,  con- 
ducting cost-effectiveness  and  cost-benefit  anal- 
yses (CEA/CBAs) — is  a  natural  next  step  in  this 
research  process.  To  conduct  such  CEA/CBAs 
encourages  the  explicit  analysis  of  the  resources 
used  in  psychotherapy  and  the  effects  (positive 
and  negative)  of  different  resource  allocation 
decisions. 

Potentially,  CEA/CBA  is  a  set  of  procedures 
that  can  aid  in  decisionmaking  about  the  use  of 
psychotherapy.  Increasingly  severe  economic 
constraints,  as  well  as  the  call  for  an  expansion 
of  mental  health  services  (see  219),  make  it  espe- 
cially important  to  understand  how  the  effects 
of  psychotherapy  are  related  to  the  resources  it 
consumes.  There  are  a  great  number  of  compet- 
ing pressures  for  health  care  resources,  and, 
ideally,  CEA/CBA  applied  to  psychotherapy 
can  serve  as  an  aid  to  resolving  these  conflicts. 
Although  the  application  of  CEA/CBA  is,  per- 
haps, not  as  well  developed  in  psychotherapy  as 
in  other  areas  (see  203),  the  current  policy  con- 
troversy about  mental  health  treatments  has 
created  increased  interest  in  its  use.  The  present 
chapter  describes  the  methods  underlying  the 
conduct  of  CEA/CBA  in  psychotherapy  and  in- 
dicates both  the  potential  for  its  use  and  the 
problems  associated  with  its  application  to  men- 
tal health  treatments. 

It  is  important  to  note  that  many  of  the  issues 
of  CEA/CBA  of  psychotherapy  are  closely  re- 


lated to  the  problems  of  assessing  efficacy.  Eco- 
nomic analyses  of  psychotherapy  are  dependent 
on  the  quality  of  research  data  pertaining  to 
psychotherapy's  effects.  The  unique  problems  of 
CEA/CBA  of  psychotherapy  have  to  do  with 
the  difficulty  of  comprehensively  assessing  and 
valuing  the  effects  of  psychotherapy  (see  98, 
157,206).  Such  effects  include  the  reduction  of 
pain  and  suffering  and  enhancement  of  "well- 
being."  These  effects  are  very  difficult  to  meas- 
ure and  even  more  difficult  to  value  in  monetary 
terms.  This  difficulty  may  result  in  CBAs  of 
psychotherapy  consistently  undervaluing  the 
benefits  of  psychotherapy.  CEA,  in  contrast  to 
CBA,  does  not  require  that  such  effects  be  ex- 
pressed in  monetary  units,  but  does  rest  on  the 
premise  that  they  can  be  valued  in  some  man- 
ner. Such  difficulties  restrict  the  usefulness  of 
cost  analyses. 

The  methodological  issues  involved  in  the  de- 
velopment of  CEA/CBAs  of  psychotherapy  are 
described  in  the  following  sections.  For  a  more 
complete  description  of  CEA/CBA,  the  reader 
should  consult  OTA's  main  report  on  CEA 
(203). 1  The  present  analysis  begins  with  a  dis- 
cussion of  the  methods  for  assessing  costs,  and 
that  is  followed  by  a  discussion  of  the  methods 
for  assessing  benefits.  In  the  third  section,  the 
actual  conduct  of  cost  analyses  is  described.  The 
discussion  below  emphasizes  the  relationship  of 
CEA/CBA  to  efficacy  assessments  and  the  use- 
fulness of  CEA/CBA  in  aiding  policymaking 
about  psychotherapy. 


Hn  addition  to  the  main  report  of  OTA's  assessment  of  CEA, 
Background  Paper  #1:  Methodological  Issues  and  Literature  Re- 
view describes,  in  detail,  the  use  of  CEA/CBA  methods. 
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COST  ASSESSMENT 

For  the  purpose  of  CEA/ CBA,  the  cost  of  psy- 
chotherapy may  be  conceptualized  as  the  value 
of  various  resources  consumed  in  the  process  of 
therapy  (see  6,157,173,237).  These  costs  include 
the  value  of  a  variety  of  resources  used  to  pro- 
vide treatments,  such  as  the  value  of  the  thera- 
pist's time  and  the  value  of  the  use  of  the  treat- 
ment facility.  They  may  also  include  the  value 
of  the  patient's  time.  Inevitably,  decisions  are 
required  about  whether  to  include  or  exclude 
costs  and  how  to  value  resources  appropriately. 
These  decisions  often  reflect  the  subjective  judg- 
ments of  different  interest  groups  involved  in 
the  cost  analysis  (see  203,277)  and  the  purposes 
for  which  the  analysis  is  being  conducted.  Just 
as  different  theoretical  perspectives  on  psycho- 
therapy may  result  in  different  measures  of  ther- 
apeutic effectiveness,  so  too  may  different  per- 
spectives yield  alternative  ways  of  defining  ther- 
apy costs. 

General  Considerations 

There  are  a  number  of  general  considerations 
relevant  to  assessing  the  costs  of  resources  used 
in  psychotherapy.  Some  of  these  general  issues 
have  to  do  with  the  data  that  are  used  in  devel- 
oping cost  estimates.  Others  concern  the  use  of 
procedures  to  transform  available  data  into  use- 
ful indices  of  resources  consumed.  In  a  later  sec- 
tion, the  application  of  these  procedures  to  the 
collection  of  specific  cost  data  is  described. 

Accounting  Methods. — The  most  readily  ac- 
cessible source  of  data  to  assess  the  costs  of 
psychotherapy  are  the  entries  in  the  accounting 
records  of  a  treatment  facility  (e.g.,  a  practi- 
tioner's office,  a  mental  health  center).  Because 
they  are  usually  highly  organized  and  accessi- 
ble, accounting  records  give  a  ready  definition 
and  a  reasonably  reliable  record  of  the  moneys 
required  to  deliver  therapy.  These  direct  costs 
are  the  ones  that  are  most  often  referred  to  in 
simple  cost  analyses  of  psychotherapy.  A  caveat 
that  should  be  noted,  however,  is  that  the  costs 
that  result  from  accounting  tabulations  may  not 
include  all  or  even  most  of  the  resources  used  by 
therapy.  Costs  from  such  tabulations,  therefore, 


may  not  be  an  accurate  reflection  of  the  re- 
sources consumed. 

In  addition,  a  number  of  costs  may  not  ap- 
pear in  accounting  records.  Thus,  for  example, 
volunteered  time  and  donated  facilities  may 
play  a  large,  but  unaccounted  for,  role  in  the 
provision  of  psychotherapy.  Therapies  that  at- 
tract more  volunteers  owing  to  location  or  type 
of  patient  treated  may  appear  less  costly  than 
therapies  located  in  areas  where  volunteers  are 
scarcer  (e.g.,  in  impoverished  neighborhoods) 
or  than  therapies  treating  less  "attractive"  per- 
sons (e.g.,  sex  offenders  compared  to  children 
who  are  autistic).  Resources  contributed  to  ther- 
apy by  the  family  or  others  connected  with  the 
patient  may  also  be  considerable,  but  would  not 
be  recorded  in  an  accounting  system. 

Opportunity  Cost. — A  more  correct  ap- 
proach to  assessing  the  costs  of  psychotherapy 
than  using  accounting  costs  involves  the  op- 
portunity costs  concept.  In  using  opportunity 
costs,  one  calculates  the  value  of  a  resource  as  if 
it  were  applied  to  a  best  alternative  use.  For 
psychotherapy,  determining  opportunity  costs 
allows  one  to  consider  more  completely  the 
value  to  society  of  various  resources  consumed 
by  psychotherapy  treatments  and  significantly 
alters  the  analysis  (25,40,283,305).  Considera- 
tion of  opportunity  costs  avoids  problems  cre- 
ated by  different  accounting  procedures  and,  at 
least  conceptually,  the  problems  due  to  the  use 
of  volunteers  and  donated  facilities. 

Discounting. — The  actual  calculation  of  op- 
portunity costs  involves  the  use  of  discounting 
procedures,  which  provide  a  present  value  of  fu- 
ture costs.  Thus,  if  costs  will  be  incurred  at  a 
future  time,  they  will  appear  to  be  less  costly  if 
valued  in  the  present.  The  discount  rate  is  usual- 
ly based  on  the  prevailing  interest  rate.  Prob- 
ably, discounting  procedures  are  more  impor- 
tant for  properly  valuing  benefits  (which  are 
more  likely  to  occur  over  time)  them  for  valu- 
ing costs.  Obviously,  though,  discounting  pro- 
cedures are  necessary  when  dealing  with  costs 
such  as  those  for  a  treatment  facility. 
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Methods  for  Cost  Assessment 

The  valuation  procedures  used  to  assess  the 
most  common  cost  elements  in  psychotherapy 
are  considered  in  this  section.  The  use  of  op- 
portunity cost  procedures  and  the  complete 
assessment  of  relevant  cost  elements  are  empha- 
sized. Also  discussed  is  the  assignment  of  costs 
to  specific  treatment. 

Personnel  Costs. — The  cost  of  personnel  is 
usually  the  largest  therapy  cost  (between  60  and 
80  percent,  according  to  Levin  (157)).  Personnel 
costs  can  usually  be  estimated  simply  by  multi- 
plying the  sum  of  salaries  and  benefits  of  the 
personnel  employed  in  the  therapy  process  by 
the  time  involved.  Personnel  may  include  pro- 
fessional therapists  and  paraprofessionals,  as 
well  as  support  staff.  In  cases  where  salary  data 
are  difficult  to  analyze  or  where  time  is  volun- 
teered (e.g.,  a  therapist  donates  time  in  a  teach- 
ing facility),  personnel  costs  can  be  estimated  by 
other  means.  Thus,  for  example,  hours  of  ther- 
apy can  be  multiplied  by  standard  hourly  salary 
figures. 

Several  "time  accounting  systems"  have  been 
specifically  developed  to  collect  personnel  cost 
data  in  mental  health  treatment  facilities  (30, 
150,205).  Some  of  these  data  are  available  in  ac- 
counting records,  but  oftentimes  they  are  not 
available  in  sufficient  specificity  (i.e.,  broken 
down  by  tasks).  In  the  time  accounting  system 
described  by  Carter  and  Newman  (30),  all  per- 
sonnel, including  volunteers,  patients,  and 
salaried  staff,  were  required  to  keep  daily  rec- 
ords of  time  spent  in  therapy-related  activities. 
This  information  formed  the  basis  of  a  cost- 
accounting  system. 

Facilities  and  Equipment  Costs. — Also  in- 
cluded in  the  valuation  process  are  the  costs  of 
operating  facilities,  most  correctly  expressed  in 
terms  of  market  rent,  plus  overhead  such  as 
cooling  and  building  maintenance  costs.  Market 
rent  is  used  in  order  to  value  the  resource  in 
terms  of  its  "opportunity  cost,"  discussed 
above.  This  procedure  corrects  for  the  problem 
of  valuing  donated  facilities  and  for  valuing 
government  facilities  which  may  be  leased  (to 
the  treatment  agency)  at  artificially  low  rents. 
The  procedure  would  include  the  estimated  val- 


ue of  donated  or  loaned  rooms  and  buildings.  It 
would  also  adjust  the  rent  or  mortgage  to  assess 
their  true,  rather  than  paid-for,  value. 

Costs  for  the  equipment  and  materials  used  in 
psychotherapy  also  have  to  be  calculated.  These 
costs,  which  are  usually  available  in  accounting 
records,  include  such  resources  as  office  sup- 
plies, food,  laundry,  and  telephones.  They  may 
also  include  the  value  of  such  specialized  mate- 
rials as  psychological  tests  and  computer  scor- 
ing services.  Usually,  the  difficulty  with  assess- 
ing the  cost  of  these  specialized  resources  is  the 
lack  of  records  as  to  their  use. 

Other  Costs. — In  addition  to  personnel  and 
facilities/equipment  costs,  some  analysts  in- 
clude the  costs  of  the  patient's  time  and/ or  the 
costs  of  therapy  to  a  patient's  family.  Although 
such  costs  can,  alternatively,  be  considered  as  a 
negative  benefit  (i.e.,  subtracted  from  benefits), 
it  is  sometimes  useful  for  them  to  be  included 
with  the  actual  costs  of  providing  therapy. 
Thus,  for  example,  when  a  patient  loses  time 
from  work  activities  or  when  an  employer  has 
to  give  release  time  to  a  patient  for  therapy,  it 
may  be  useful  to  value  the  patient's  time  and 
consider  it  as  a  cost. 

Assessment  of  the  costs  of  the  patient's  time  is 
similar  to  the  way  other  personnel  costs  are  cal- 
culated. Usually,  an  accounting  can  be  be  made 
of  the  amount  of  time  that  the  patient  spends  in 
therapy.  Assuming  that  this  time  could  be  used 
productively,  it  is  multiplied  by  the  patient's 
salary.  Parallel  calculations  can  be  made  for 
family  members  who  become  involved  in  the 
therapy  or  are  required  to  spend  time  with  a  pa- 
tient as  a  result  of  therapy.  One  problem  with 
this  aspect  of  the  costing  process  is  the  problem 
of  equity,  since  some  groups  of  people  earn 
more  than  others  and,  thus,  their  time  could  be 
valued  more  highly. 

While  a  number  of  other  costs  could  be  in- 
cluded, such  as  the  psychological  cost  of  ther- 
apy, there  is  no  agreement  as  to  whether  they 
should  be  included  or  how  they  can  be  valued 
(see,  e.g.,  126,172).  The  development  of  psy- 
chological cost  measures  would  involve  the 
assessment  of  the  suffering  or  pain  of  a  patient 
as  a  result  of  therapy.  However,  such  costs,  as 
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well  as  the  costs  of  mental  illness  are  usually 
considered  as  a  reduction  in  the  benefit  of  a 
treatment. 

Specifying  Costs. — For  a  simple  CEA  or 
CBA,  the  sum  of  personnel,  facilities,  equip- 
ment, and  materials  costs  may  be  adequate.  For 
an  analysis  which  seeks  to  identify  the  proce- 
dures, therapists,  patient  types,  or  settings  that 
consume  more  costs  than  others,  however,  pro- 
cedures have  to  be  devised  for  assigning  costs  to 
individual  components  of  therapy.  Analyses  of 
this  type  may  become  even  more  complex  when 
distinctions  have  to  be  made  among  various 
types  of  costs  (157,248). 

Cost  data  that  can  be  used  for  analyses  of  the 
resources  used  for  specific  components  of  ther- 
apy can  be  collected  either  during  therapy  or 
after  it  is  completed.  If  collected  after  therapy, 
the  analysis  requires  summary  data  to  be 
broken  down  into  different  costs  corresponding 
to  therapy  components.  Because  many  assess- 
ments of  psychotherapy  have  been  introduced 
only  after  a  therapy  program  has  begun,  the 
breakdown  method  has  received  considerable 
attention.  Analyses  that  examine  the  cost  effec- 
tiveness of  treating  individual  patients  may 
divide  overhead  costs  such  as  salary,  rent,  and 
basic  supplies  equally  among  patients,  accord- 
ing to  the  amount  of  their  therapy  (e.g.,  number 
of  therapy  hours). 

Some  cost  assessment  procedures  assign  ther- 
apist costs  to  patients  according  to  the  amount 
of  time  that  therapists  spend  working  with  dif- 
ferent patients;  then,  the  costs  of  overhead,  per- 
sonnel, and  other  resources  are  divided  equally 
across  patients  (e.g.,  30,149,199,305).  To  com- 
pare the  relative  cost,  cost  effectiveness,  or  cost 


benefit  of  different  components  of  therapy  pro- 
grams, the  difference  in  direct  costs  of  the  var- 
ious treatment  components  is  calculated.  In 
such  analyses,  the  overhead  costs  that  are  the 
same  for  each  component  are  ignored.  These 
direct  costs  are  then  used  in  CEA/CBAs,  as  de- 
scribed below. 

Discussion  of  Cost  Assessment 

Despite  some  longstanding  interest  in  assess- 
ing the  cost  of  psychotherapy  (e.g.,  76)  and  the 
apparent  ease  of  applying  standard  valuation 
methods  to  psychotherapy,  the  assessment  of 
psychotherapy's  cost  is  not  widespread,  nor 
has  such  assessment  been  evaluated.  Certainly, 
techniques  for  the  measurement  of  effectiveness 
are  much  better  developed  than  those  for  meas- 
uring costs.  Most  psychotherapy  research — per- 
haps 95  percent — neglects  the  cost  of  the  treat- 
ment. The  implications  of  this  neglect  for  CEA/ 
CBAs  of  psychotherapy  are  important  to  con- 
sider (16).  It  should  be  noted  for  example,  that 
no  standards  as  to  what  should  be  included  in 
cost  analyses  have  been  developed.  More  im- 
portantly, the  available  studies  may  reflect  a 
narrow  range  of  treatments  (probably  those 
treatments  which  are  either  very  costly  or  very 
low  in  cost).  In  addition,  cost  data  may  be 
derived  from  existing  studies  even  if  not  explic- 
itly included  in  the  original  analysis.  Usually, 
the  available  information  on  numbers  of  thera- 
pists, patients,  and  treatment  length  is  detailed 
enough  for  rough  cost  estimates  to  be  made. 
Combined  with  secondary  analysis  procedures, 
the  availability  of  these  data  may  provide  a 
promising  opportunity  for  further  R&D  of  psy- 
chotherapy CEA/CBAs. 


BENEFIT  ASSESSMENT 

The  valuation  of  "benefits"  resulting  from 
psychotherapy  which  can  be  used  in  CBA,  un- 
fortunately, is  even  more  problematic  than  cost 
assessment  (e.g.,  140,237).  The  translation  of  ef- 
fects into  benefits  is  necessary  to  provide  a  com- 
mon metric  for  comparing  resources  used  with 
effects.  The  problems  of  benefit  assessment  have 


to  do  with  selecting  effects  to  be  valued  and 
determining  appropriate  ways  of  translating  ef- 
fects into  benefits  (i.e.,  valuing  effects  in  mone- 
tary terms).  Below,  these  problems  are  dis- 
cussed in  terms  of  the  types  of  benefits  produced 
by  psychotherapy — to  patients,  to  those  asso- 
ciated with  patients,  and  to  society. 


Ch,  5— Measuring  the  Cost  Effectiveness  and  Cost  Benefit  of  Psychotherapy  •  55 


Benefits  to  Patients. — The  most  obvious  ben- 
efits of  psychotherapy  accrue  directly  to  the  pa- 
tient, although  different  effects  of  therapy  have 
somewhat  different  problems  attached  to  their 
valuation.  Thus,  while  a  change  in  earnings 
may  be  valued  in  a  relatively  straightforward 
way,  other  benefits  of  psychotherapy,  such  as  a 
reduction  in  pain  or  anxiety,  are  more  difficult 
to  measure  and  value.  Because  these  intangibles 
are  so  difficult  to  quantify  in  monetary  units, 
any  CBA  of  psychotherapy  may  undervalue  the 
benefits.  It  is  also  possible  that  some  of  the 
negative  benefits  of  psychotherapy  (perhaps, 
the  effects  of  stigmatization  of  being  a  patient) 
will  not  be  calculated.  To  avoid  this  problem, 
many  studies  separately  analyze  tangible  and 
intangible  benefits,  comparing  only  tangible 
benefits  to  costs. 

From  the  perspective  of  some  economists 
(e.g.,  251),  it  is  argued  that  a  patient's  will- 
ingness to  pay  for  psychotherapy  reflects  its 
value  to  the  patient.  Thus,  that  which  patients 
are  willing  to  pay  for  therapy  is  the  net  value  of 
the  expected  health,  social,  and  economic 
benefits,  minus  the  psychological  suffering,  lost 
time,  and  personal  costs  that  are  incurred  as  a 
result  of  undergoing  psychotherapy.  Underlying 
the  use  of  the  "willingness-to-pay"  concept  is  the 
assumption  that  the  patient  has  made  an  in- 
formed decision  to  pay  the  required  fee.  How- 
ever, the  willingness-to-pay  assumption,  while 
theoretically  possible,  assumes  that  the  mentally 
distraught  person  has  made  a  rational  decision. 
Since  such  individuals  seldom  have  access  to 
clear  information  about  psychotherapy's  costs 
and  benefits,  this  assumption  is  probably  un- 
likely and  has  not  really  been  applied. 

Some  economists,  for  example,  Weisbrod 
(296),  recognizing  the  inherent  problem  with  ap- 
plying the  willingness-to-pay  criterion  to  psy- 
chotherapy, have  proposed  more  direct  assess- 
ment of  patient  benefits.  In  Weisbrod's  research, 
the  variety  of  effects  of  treatment  are  identified 
and,  to  the  extent  possible,  transformed  into 
monetary  values.  For  some  effects,  such  as  im- 
proved quality  of  life  or  absence  of  mental  ill- 
ness, no  pecuniary  value  can  be  assigned.  These 
factors  are  therefore  not  included  in  an  assess- 


ment of  benefits,  but  can  be  contrasted  with  the 
costs  of  particular  therapies. 

Aside  from  such  problems  of  valuing  certain 
effects,  there  is  also  a  problem  of  identifying 
which  effects  should  be  attributed  to  psycho- 
therapy. This  is  accomplished  by  comparing 
treatment  and  control  group  data.  To  the  extent 
valid  control  group  data  are  not  available,  re- 
sulting benefit  estimates  may  be  in  error.  While 
the  availability  of  appropriate  effectiveness  data 
is  a  problem  for  the  assessment  of  any  type  of 
benefit,  it  is  a  particular  problem  for  assessing 
the  effects  on  a  patient.  For  most  effects,  there 
will  be  a  variety  of  other  possible  causes  which 
will  be  difficult  to  separate  without  control 
group  data. 

Benefits  to  Those  Associated  With  Patients. — 

The  effects  of  psychotherapy  may  extend  be- 
yond the  patient.  Thus,  the  family  and  friends 
of  a  patient  may  have  their  own  quality  of  life 
improved  if  therapy  is  successful.  In  a  more  tan- 
gible way,  they  may  also  achieve  more  produc- 
tivity in  their  own  work  and  have  more  time 
available  for  their  own  needs.  The  opportunity 
value  of  their  improved  productivity  and  time 
can  be  calculated  and,  where  appropriate,  con- 
sidered as  a  benefit  of  psychotherapy. 

As  a  result  of  therapy,  patients  may  also  be 
more  productive  workers,  and  the  benefits  of 
this  productivity  may  accrue  to  their  employer 
(over  and  above  the  wages  paid  to  the  employ- 
ee). Absenteeism  may  be  reduced,  accidents  be 
fewer,  and  a  host  of  other  benefits  are  poten- 
tially the  result  of  psychotherapy  (e.g.,  138). 
These  benefits,  of  course,  must  be  reduced  by 
costs  for  an  employer  to  provide  psychotherapy 
or  to  allow  employees  release  time  to  undergo 
psychotherapy.  As  noted  earlier,  however, 
these  costs  can  either  be  considered  a  direct  cost 
or  subtracted  from  the  benefits.  It  is  important 
to  ensure  that  an  analysis  uses  consistent  proce- 
dures. It  is  also  important  that  the  same  benefits 
are  not  counted  twice;  thus,  for  example,  the 
analyst  must  be  careful  not  to  count  the  same 
wages  as  a  benefit  to  the  patient  and  to  the 
employer. 

Benefits  to  Society. — Some  of  the  most  tangi- 
ble benefits  of  psychotherapy  may  accrue  to  so- 
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ciety.  Thus,  the  maintenance  of  employment  or 
a  reduction  in  criminal  activities  may  yield  a 
savings  directly  to  society.  These  effects,  re- 
flected in  such  savings  as  reduced  unemploy- 
ment payments,  may  be  over  and  above  the 
benefits  to  patients  and  employers.  Although 
such  outcomes  may  be  relatively  easy  to  value, 
a  problem  is  that  the  benefits,  if  they  exist, 
probably  accumulate  over  a  long  period  of  time. 
In  most  cases,  these  benefits  would  have  to  be 
very  large  to  offset  the  impact  of  discounting 
their  value  over  the  time  they  are  received. 

It  should  be  noted  that  most  often  the  benefits 
described  above  will  be  in  the  form  of  expected 
cost  savings.  Therapy-related  benefits,  such  as 
reductions  in  work  absenteeism,  physician 
visits,  drug  abuse,  and  arrests,  have  each  been 
considered  in  CEA/CBA  studies  (e.g.,  51,228, 
244,245).  In  these  studies,  the  cost  of  each  unit 
of  service  is  estimated  from  average  rates  or 
from  accounting  records,  and  the  reduction  in 
use  of  units  of  social  services  is  multiplied  by  the 


unit  cost  to  estimate  monetary  cost-savings  ben- 
efits. Usually,  the  validity  of  control  group  data 
to  estimate  these  savings  is  critical  in  order  to 
separate  the  effects  of  psychotherapy  on  these 
variables  from  other  causes. 

Discussion. — In  several  ways,  developing 
benefit  measures  is  more  difficult  than  the  com- 
parable procedures  for  cost  assessment.  Because 
of  the  problems  associated  with  using  willing- 
ness to  pay  as  the  basis  for  valuing  benefits, 
methods  have  to  be  developed  to  transform  ef- 
fects into  benefits.  If  errors  are  made,  they  will 
probably  result  in  some  analyses'  not  taking 
benefits  (especially  the  psychological  ones)  into 
account  and,  thus,  understating  the  benefits  of 
therapy.  In  addition,  it  is  probably  easy  to  err 
by  not  including  some  benefits.  Because  benefits 
potentially  accrue  to  a  large  number  of  people 
and  societal  agencies,  often  more  so  than  the 
number  of  people  or  groups  who  incur  costs, 
problems  in  analyses  may  result. 


METHODS  FOR  CEA/CBA 

The  purpose  of  carefully  measuring  the  costs, 
as  well  as  the  benefits,  of  psychotherapy  is  to  be 
able  to  conduct  CEAs  and  CBAs.  CEA  differs 
from  CBA  and  it  is  essential  to  recognize  the  dif- 
ference (see  203).  Cost-effectiveness  studies  re- 
quire only  that  the  costs  of  psychotherapy  be 
valued  in  dollars.  In  CEA,  the  effects  are  not 
valued  in  monetary  terms  and  can  be  expressed 
in  any  units.  In  contrast,  CBA  requires  that 
both  the  costs  and  outcomes  be  valued  in  mone- 
tary terms.  (Theoretically,  costs  and  benefits 
need  only  be  expessed  in  the  same  unit  for  CBA, 
but  this  unit  is  nearly  always  monetary.)  It 
should  also  be  noted  that  informal  cost  analyses 
can  be  conducted  where  a  researcher  selectively 
pays  attention  to  some  aspects  of  resource  use 
or  benefits. 

Although  CEA  usually  requires  a  simpler  set 
of  procedures  and  yields  less  comparative  in- 
formation than  CBA,  CEA  is  often  considered  a 
more  appropriate  tool;  in  other  cases,  when  cer- 
tain comparative  information  is  necessary  and 
when  valuation  problems  can  be  overcome, 


CBA  is  better  suited  to  the  problem  (see,  e.g., 
98,157,206,305,307).  In  still  other  cases,  when 
comprehensive  data  are  not  available,  actual 
CEA/CBA  is  not  done,  but  the  costs  and  bene- 
fits are  compared  informally.  Most  commonly, 
CEA  and  CBA  have  been  used  to  compare  the 
"worth"  of  different  psychotherapies  as  pro- 
vided in  different  settings.  Also  frequently  em- 
ployed are  formal  and  informal  CEA/CBA 
studies  designed  to  yield  information  to  those 
directly  responsible  for  the  therapy.  Such  anal- 
yses often  try  to  assign  costs  and  outcomes  to 
specific  treatment  processes,  so  that  the  optimal 
mixture  of  processes  can  be  obtained  (see  305). 
Below,  some  procedures  associated  with  the 
conduct  of  CEA  and  CBA  studies  are  described. 

Cost-Effectiveness  Analysis 

When  benefits  and  costs  cannot  be  valued  in 
the  same  units,  or  when  the  outcomes  of  a  treat- 
ment seem  more  valid  when  expressed  in  their 
natural  units  (e.g.,  reduction  of  a  anxiety),  then 
CEA  is  probably  more  appropriate  than  CBA 
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(e.g.,  157,237).  In  addition,  when  treatments 
are  compared  that  have  similar  goals,  CEA  may 
provide  an  adequate  and  appropriate  methodol- 
ogy. In  conducting  CEA,  outcome  data  are  di- 
vided by  costs  to  form  cost-effectiveness  ratios. 
The  procedure  allows,  for  example,  the  compar- 
ison of  several  therapies  to  determine  which 
therapy  produces  the  greatest  amount  of  change 
for  the  least  cost.  A  variety  of  examples  of  this 
procedure  are  given  in  chapter  6. 

Tabulation  and  Matrix  Methods. — One  of  the 
simplest  ways  to  analyze  effectiveness  and  cost 
relationships  is  to  display  the  data  in  an  array. 
A  simple  tabulation  model  described  by  Krum- 
boltz  (150)  provides  a  rudimentary  example  of 
how  the  basic  direct  cost,  processes,  and  effec- 
tiveness can  be  arrayed  in  a  useful  way.  A  table 
is  developed  with  the  sideheadings  "Problem 
Identification,"  "Method,"  and  "Outcome."  Un- 
der a  supraheading,  "Cost,"  are  the  headings 
"Activity,"  "Hours,"  and  "Dollars."  Cost  break- 
downs, problems  to  be  worked  on  further,  and 
the  other  essential  information  for  simple  anal- 
ysis are  contained  here. 

Newman  (30,198)  has  developed  a  CEA  pro- 
cedure that  has  been  adopted  in  a  number  of 
psychotherapy  settings.  An  instrument  is  used 
to  measure  level  of  functioning  along  a  range  of 
dimensions.  Newman  arrays  his  outcome  data 
in  a  matrix,  using  dimensions  such  as  level  of 
functioning  before  treatment  and  level  of  func- 
tion after  therapy.  The  cells  of  the  matrix  are 
completed  with  the  number  of  patients  who 
functioned  at  that  level  before  treatment  and 
who  moved  to  another  level  of  functioning  by 
the  end  of  therapy  (or  who  stayed  in  the  same 
level).  Next,  the  cost  of  treating  each  patient 
in  that  cell  of  the  matrix  is  summed,  and  divided 
by  the  number  of  patients  whose  functioning 
change  is  described  by  the  cell.  The  resulting 
cost-per-patient  ratio  reflects  cost  effective- 
ness in  terms  of  its  position  in  the  matrix  (see 
also  270). 

Linear  Functions. — To  describe  and  predict 
relationships  between  effectiveness  and  cost,  it 
is  possible  to  develop  equations  that  describe 
cost-effectiveness  relationships  (see  305).  The 
techniques  that  would  achieve  a  given  level  of 
effectiveness  also  can  be  chosen  from  this 


graphic  model  of  the  cost-effectiveness  relation- 
ship. Not  only  can  effectiveness  be  predicted, 
but  the  effectiveness  of  particular  techniques  for 
different  costs  can  be  determined.  In  an  actual 
situation,  a  number  of  possibilities  exist  for  a 
function  to  describe  this  relationship  (e.g., 
linear  or  exponential).  Ideally,  if  this  method 
were  chosen  before  therapy,  several  plausible 
models  would  be  chosen,  such  as  the  linear  and 
exponential,  and  the  average  of  predictions 
generated  from  them  would  be  used  in  decisions 
until  further  information  supported  one  model 
over  the  other. 

Linear  Programing. — Finally,  some  applica- 
tions of  CEA  techniques  attempt  to  incorporate 
information  on  cost  limits,  as  well  as  on  the  fac- 
tors that  determine  the  effectiveness  of  psycho- 
therapy. The  basic  concept  underlying  linear 
programing  is  to  consider  not  only  the  factors 
that  contribute  most  to  therapy  outcomes,  but 
also  the  cost  of  less  effective  factors  and  budget 
restrictions.  Linear  programing  is  a  statistical 
procedure  used  to  find  the  exact  mixture  of  the 
most  contributory  factors  that  are  possible 
within  budget  constraints. 

The  equations  for  linear  programing  bring 
together  information  on  which  therapy  tech- 
niques, delivery  systems,  or  therapists  work 
best  and  on  the  amounts  of  each  resource 
needed  to  implement  each  technique  or  delivery 
system,  or  to  hire  each  therapist.  Equations  can 
be  used  to  minimize  the  total  costs  of  achieving 
a  prescribed  degree  of  effectiveness  or  benefit. 
The  equations  can  also  be  manipulated  further 
to  discover  which  cost  constraint  could  be  fitted 
to  yield  the  maximum  improvement  of  effective- 
ness or  benefit.  A  number  of  psychotherapy  re- 
searchers have  advocated  use  of  these  and  re- 
lated techniques  to  conduct  cost  analyses  in  the 
human  services  delivery  such  as  psychotherapy 
(e.g.,  1,14,72,115,193,310). 

Cost-Benefit  Analysis 

In  CBA,  benefits  are  summed  using  the  same 
units  (e.g.,  dollars,  person-hours)  and  costs  are 
summed  using  the  same  units  as  benefits.  A 
ratio  is  then  derived  by  dividing  total  benefits 
by  total  costs.  If  benefits  exceed  costs,  the  ratio 
is  larger  than  1,  and  if  benefits  are  less  than 
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costs,  the  ratio  is  less  than  1.  Such  benefit/cost 
ratios  provide  a  convenient  economic  index  of 
the  net  benefit  of  an  activity. 

One  reason  that  CBA  may  be  more  useful 
than  CEA  is  that  decisions  which  compare  ben- 
efits to  costs  of  a  treatment  program  are  seldom 
made  in  isolation.  Often,  benefits  and  costs  of  a 
given  treatment  are  most  useful  when  compared 
with  benefit/cost  ratios  of  other  treatments 
competing  for  the  same  funds.  Thus,  an  alterna- 
tive treatment  program  may  be  available  which 
could  generate  superior  benefit-to-cost  ratios. 
The  important  factors  are  the  difference  in  the 
costs  of  alternative  treatments,  the  difference  in 
benefits  of  alternative  treatments,  and  the  ratio 
of  benefit  to  cost.  An  argument  can  be  made 
that  effectiveness  measures  should  be  used,  in 
some  cases,  instead  of  benefit  measures,  because 
effectiveness  data  probably  retain  more  ac- 
curate and  valid  information  on  treatment  out- 
comes. That  very  much  depends  on  the  situ- 
ation, however,  and  it  may  be  necessary  to  pre- 
sent both  types  of  data. 

To  illustrate  the  problem,  consider  the  appli- 
cations of  CEA  methods  to  psychotherapy  by 
Halpern  and  his  colleagues  (22,113,114).  Their 
approach  views  improvement  in  therapist  rat- 
ings of  patient  functioning  as  a  monetizable  in- 
crement in  the  economic  value  of  the  patient. 
This  increment  is  contrasted,  in  simple  benefit/ 
cost  ratios,  to  the  monetary  cost  of  treating  the 
patient.  Potter,  Binner,  and  Halpern  (216)  have 
made  their  models  somewhat  more  sophisti- 
cated by  considering  benefits  according  to  the 
amount  of  time  the  patient  stays  in  the  com- 
munity as  well  as  the  improvement  noted  by  the 
end  of  therapy.  Although  perhaps  useful,  this 
method  suffers  from  potential  bias  in  therapist 
ratings  of  improvement  in  patient  functioning 
and  in  assignment  of  a  somewhat  arbitrary 
value  (e.g.,  $10,000)  to  each  functioning  unit 
improvement.  It  may  be  more  accurate  to  use 
the  actual  functioning  unit  scores  in  a  CEA. 

Depite  such  problems,  there  are  still  reasons 
to  prefer  benefit  measures,  especially  when  the 
units  in  which  benefits  have  been  measured  are 
more  directly  meaningful  than  those  used  by 
Halpern,  et  al.  (22).  Ratios  of  effectiveness  di- 
vided by  cost  may  not  provide  as  much  infor- 


mation as  benefit  divided  by  marginal  cost,  be- 
cause if  the  latter  is  greater  than  1,  the  addi- 
tional benefits  of  one  of  the  alternative  treat- 
ments can  be  said  to  be  "worth"  the  additional 
costs  it  requires.  It  is  difficult  to  make  a  similar 
statement  about  effectiveness /cost  ratios. 

Fishman's  research  (77,78),  developed  to  as- 
sess the  effects  of  a  community  mental  health 
center,  illustrates  the  use  of  similar  CBA  proce- 
dures. Fishman  views  CBA  as  an  experiment  in 
which  outcome  and  cost  data  are  gathered  using 
the  research  designs  for  assessing  efficiency 
described  earlier.  If  the  effectiveness  of  one  pro- 
gram or  program  component  is  shown  by  statis- 
tical analysis  to  be  significantly  superior  to  the 
effectiveness  or  benefit  of  another  program,  and 
the  costs  of  the  two  do  not  differ  significantly, 
then  the  program  with  superior  effectiveness  is 
more  cost  effective.  If  the  costs  of  alternative 
programs  differ  significantly,  but  their  effec- 
tiveness or  benefit  does  not,  then  the  least  costly 
program  is  adopted.  Fishman  acknowledges 
that  his  model  breaks  down  in  situations  where 
the  significantly  more  effective  or  beneficial 
program  is  also  significantly  more  costly.  The 
question  as  to  whether  the  increment  in  effec- 
tiveness or  benefit  is  "worth"  the  increment  in 
cost  is,  as  noted  earlier,  a  question  for  marginal 
benefit-cost  analysis. 

Net  Benefit  Analysis. — It  should  be  recog- 
nized that  ratios  of  benefit  to  cost  or  effec- 
tiveness to  cost  do  not  really  yield  information 
about  the  absolute  amount  of  benefits  and  total 
costs  involved.  This  information  may  be  impor- 
tant in  decisions  that  must  deal  with  limits  on 
the  maximum  cost  allowable  and  on  the  mini- 
mum benefit  that  should  be  produced.  The 
amount  that  benefits  exceed  costs  also  may  be  of 
concern.  A  benefit/cost  ratio  of  2  can  be  pro- 
duced by  a  benefit  of  $200  and  a  cost  of  $100  for 
one  program,  or  by  a  benefit  of  $200,000  and  a 
cost  of  $100,000  for  another  program,  but  these 
two  programs  are  not  the  same.  If  no  more  than 
$2,000  can  be  spent,  the  former  treatment  pro- 
gram is  the  only  feasible  one;  if  benefits  must  ex- 
ceed $2,000,  then  the  second  treatment  program 
is  the  only  one  possible.  To  aid  in  maximizing 
benefits,  analysts  often  calculate  net  benefits, 
which  are  present-valued  benefits  minus  pres- 
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ent-valued  costs.  This  is  one  of  the  advantages 
in  expressing  outcomes  and  costs  in  the  same 
units.  If  the  net  benefit  is  negative,  then  the  pro- 
gram is  not  worthwhile;  if  net  benefits  exceed 
zero,  the  program  is  worthwhile.  This  informa- 
tion may  be  more  useful  than  ratios  in  many  in- 
stances, although  it  might  also  be  desirable  to 
consider  calculations  of  net  benefit  per  patient. 

Sensitivity  Analyses. — In  any  CBA,  it  is  im- 
portant to  consider  the  impact  of  alternatively 
valuing  benefits  and  the  impact  of  measurement 
error.  For  example,  if  a  benefit  of  $100,000  is  in 
error  by  ±  10  percent  and  the  cost  of  $90,000  in 
error  by  ±  10  percent,  the  benefit/cost  ratio 
might  vary  from  0.91  ($90,000)/($99,000),  to 
1.36  ($110,000)/($81,000),  instead  of  the  1.11 
($100,000)/($90,000)  calculated  originally.  Pro- 
viding these  alternative  calculations  would  give 
the  interpreter  of  the  CBA  an  idea  of  the  possi- 
ble error. 


Statistical  analyses  can  also  be  applied  to  such 
problems.  Thus,  for  example,  one  can  calculate 
standard  error  scores  which  provide  a  precise 
statistical  measure  of  error  (see,  e.g.,  201). 
Statistical  procedures  can  also  be  used  to  test  the 
significance  of  different  cost/benefit  ratios  (30). 
These  ratios  are  typically  calculated  for  the 
treatment  as  a  whole,  in  which  case  possible 
error  in  measurements  of  costs,  effectiveness, 
and  benefits  cannot  be  treated  as  variance  about 
a  mean,  but  instead  as  an  absolute  error.  In  such 
analyses,  the  degree  to  which  measurement 
error  may  influence  the  benefit/cost  ratio  can  be 
investigated  only  by  first  establishing  a  reason- 
able range  of  possible  error  and  next  calculating 
(benefit  given  error) /(highest  cost  given  error), 
and  (highest  benefit  given  error)/(lowest  cost 
given  error).  Some  of  these  procedures  are  also 
applicable  to  CEA. 


SUMMARY 

There  exist  a  variety  of  methods  to  assess  the 
costs  and  benefits  of  psychotherapy  and  to  com- 
pare the  data  generated  by  various  studies.  Al- 
though in  the  case  of  psychotherapy,  unique  dif- 
ficulties arise  (in  particular,  having  to  do  with 
the  valuation  of  benefits),  the  problems  of  con- 
ducting CEA/CBAs  are  not  necessarily  unique 
to  assessments  of  psychotherapy.  In  every  in- 
stance, the  usefulness  of  such  analyses  is  very 


much  dependent  on  the  quality  and  availability 
of  outcome  data.  It  would  seem,  however,  that 
much  more  methodological  development  needs 
to  take  place  with  respect  to  CEA/CBA  before 
these  techniques  can  be  used  with  known  reli- 
ability and  validity  in  psychotherapy  assess- 
ments. The  substantive  literature  describing  the 
application  of  these  methods  to  psychotherapy 
is  examined  in  the  next  chapter  of  this  report. 
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As  noted  in  the  previous  chapter,  the  applica- 
tion of  cost-effectiveness  and  cost-benefit  analy- 
sis (CEA/CBA)  to  psychotherapy  is  fairly  recent 
and  probably  far  less  developed  than  other 
areas  of  psychotherapy  research.  Not  surpris- 
ingly, therefore,  there  are  a  number  of  problems 
with  the  available  CEA/CBA  research.  Some  of 
these  problems  have  to  do  with  the  methodolog- 
ical limitations  of  the  ways  in  which  psycho- 
therapy's  outcomes  have  been  assessed  (see  chs. 
2  and  3);  additional  problems  relate  to  the  way 
CEA/CBA  methods  have  been  employed.  CEA/ 
CBAs  of  psychotherapy,  for  example,  focus 
solely  on  the  costs  of  treatment  and  pay  mini- 
mal attention  to  outcomes.  Others  use  single  or 
insensitive  measures  of  outcome.  Still  others  fail 
to  measure  costs  or  outcomes  comprehensively, 
and  many  CEA/CBA  studies  use  accounting 
costs,  rather  than  opportunity  costs.  CEA/CBA 
studies  that  use  only  accounting  costs  potential- 
ly underestimate  the  actual  amount  of  resources 
required  to  provide  psychotherapy  treatments 
and  the  benefits  that  may  accrue  from  their  ap- 
plication. At  present,  CEA/CBA  analyses  of 
psychotherapy  seem  to  have  neither  the  meth- 
odological rigor,  nor  the  breadth  of  application, 
to  be  centrally  useful  in  assessing  psychother- 
apy's  value  (see  2). 

Despite  the  problems  with  CEA/CBA  re- 
search, however,  it  does  not  follow  that  CEA/ 
CBA  studies  of  psychotherapy  cannot  or  should 
not  be  conducted.  As  described  in  chapter  5, 
there  are  methods  for  valuing  the  resources  used 
by  psychotherapy;  in  addition,  methods  for 
valuing  the  outcomes  of  psychotherapy,  though 
less  well  developed,  can  be  applied.  To  the  ex- 
tent that  CEA/CBA  is  a  useful  adjunct  to  the 
health  policy  decisionmaking  process  (see  203), 
its  application  to  psychotherapy  seems  as  rea- 
sonable (see,  e.g.,  179)  as  its  application  to 
other  health  problems.  The  use  of  CEA/CBA 
studies  may  provide  information  about  the  use 


and  impact  of  psychotherapy  that  is  otherwise 
unavailable.  At  the  very  least,  such  studies  may 
help  structure  the  questions  and  data  about 
psychotherapy  in  a  way  that  should  aid  policy- 
makers in  choosing  among  alternative  pro- 
grams. CEA/CBA  may  also  be  useful  in  stimu- 
lating better  research  about  the  effects  of 
psychotherapy.  When  resource-effect  relation- 
ships are  considered  as  in  CEA/CBA,  aspects  of 
psychotherapy  are  highlighted  which  otherwise 
might  be  ignored;  thus,  for  example,  cost  analy- 
ses may  suggest  assessing  a  wider  range  of  vari- 
ables than  would  usually  be  considered  theoret- 
ically relevant. 

Although  CEA/CBA  studies  of  psychother- 
apy currently  appear  to  have  limited  policy 
usefulness,  it  should  be  recognized  that  data 
from  such  studies  sometimes  provide  helpful 
policy  information.  There  are  reliable  and  valid 
data  which  indicate  that  some  conditions  of  of- 
fering psychotherapy  are  more  cost  effective  or 
cost  beneficial  than  others.  Although  these  data 
may  have  limited  generalizability,  they  may  be 
helpful  in  making  specific  resource  allocation 
decisions  (e.g.,  reimbursement  policy  for  mental 
health  treatments).  The  data  also  suggest  prom- 
ising areas  for  future  research. 

The  review  below  describes  selected  aspects 
of  a  number  of  recent  cost-effectiveness  and 
cost-benefit  studies  of  psychotherapy.  The  focus 
is  on  individual  studies,  because  syntheses  of  the 
CEA/CBA  literature  in  psychotherapy  are  not 
yet  available.  In  general,  the  available  CEA/ 
CBA  studies  do  not  compare  different  psycho- 
therapy treatments — instead,  they  deal  with  the 
factors  that  affect  the  provision  of  psychother- 
apy treatments.  Thus,  a  major  emphasis  of 
CEA/CBA  studies  conducted  to  date  has  been 
differences  among  treatment  settings.  Another 
emphasis  has  been  the  analysis  of  conditions 
under  which  either  low-cost  therapies  or  poten- 
tially high-benefit  treatments  have  been  used. 
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Because  of  the  difficulties  associated  with 
measuring  psychotherapy's  benefits  (see  ch.  5), 
the  literature  is  probably  biased  toward  assess- 
ing low-cost  treatments.  This  characteristic  of 
the  available  studies  should  be  kept  in  mind. 

The  attempt  in  the  present  review  is  not  to 
provide  a  comprehensive  analysis  of  the  CEA/ 
CBA  literature,  but  to  illustrate  the  types  of 
studies  available  and  their  findings.  A  number 
of  the  studies  reported  here  were  included,  in 
some  form  (e.g.,  were  considered  within  a  re- 
view that  was  analyzed),  in  the  discussion  of  ef- 
fectiveness in  chapter  4.  They  are  described  in 
more  detail  in  the  present  chapter  because  of 
their  explicit  consideration  of  resource  issues 
(i.e.,  assessment  of  cost  and/or  benefits).  In- 
cluded in  the  present  chapter  are  a  number  of 


studies  that  assess  effectiveness  poorly  and  con- 
tain other  methodological  flaws.  These  studies 
are  included  to  suggest  the  potential  contribu- 
tion of  CEA/CBA  research  on  psychotherapy 
and  to  identify  some  of  the  problems  of  con- 
ducting CEA/CBA. 

The  discussion  below  is  organized  according 
to  the  treatment  factors  and  patient  conditions 
to  which  CEA/CBA  has  been  applied.  First,  the 
review  considers  a  number  of  treatment-related 
characteristics  that  have  been  subjected  to 
CEA/CBA.  Next,  it  considers  therapist  factors 
as  they  relate  to  costs  of  treatment  and  out- 
comes. Finally,  several  specific  patient  condi- 
tions and  CEA/CBA  studies  of  psychotherapy 
treatments  applied  to  these  conditions  are 
reviewed. 


TREATMENT-RELATED  CHARACTERISTICS 


As  noted  above,  a  focus  of  psychotherapy 
CEA/CBAs  conducted  to  date  has  been  charac- 
teristics of  the  treatment  system  in  which  psy- 
chotherapy is  provided.  Some  studies  have  fo- 
cused on  the  provision  of  psychotherapy  under 
various  forms  of  institutional  care,  and  some 
have  compared,  for  example,  institutional  and 
outpatient/community-based  treatments.  The 
results  of  these  comparisons  are  not  easy  to 
summarize,  although  it  does  seem  clear  that  the 
conditions  of  treatment  can  vary  widely  in 
terms  of  the  resources  they  use  and  their  impact. 

Inpatient  Therapy 

One  area  in  which  CEA/CBAs  of  psychother- 
apy have  frequently  been  done  concerns  institu- 
tionalization of  mental  patients  and  the  provi- 
sion of  inpatient  psychotherapy.  The  costs  of 
institutionalizing  a  patient  are  high,  and  the 
costs  of  providing  therapy  (in  addition  to  caring 
for  the  patient)  are  potentially  even  higher.  A 
number  of  studies  have  examined  the  costs  and 
benefits  of  various  treatments  offered  patients, 
in  particular,  the  provision  of  therapy  along 
with  custodial  or  milieu  care.  The  conclusion  of 
a  number  of  these  studies  is  that  the  provision 
of  psychotherapy  can  provide  more  cost-effec- 


tive or  cost-beneficial  treatment  (e.g.,  24,130, 
175,273,289).  These  findings  were  obtained  be- 
cause therapy  may  reduce  the  amount  of  hospi- 
talization or  achieve  larger  benefits  in  some 
other  area. 

McCaffree  (175),  in  one  study  of  the  benefits 
of  "intensive"  inpatient  treament,  reviewed  the 
costs  and  results  of  treatment  in  the  Washington 
State  hospital  system  during  two  periods.  Dur- 
ing the  two  periods,  there  was  a  shift  in  the 
treatment  of  institutionalized  patients.  In  the 
first  period,  only  custodial  care  was  provided; 
in  the  second  period,  various  forms  of  psycho- 
therapy (which  was  called  "intensive  therapy") 
were  provided  as  well.  McCaffree  assessed  the 
costs  for  both  types  of  treatment.  He  included 
both  public  costs  (e.g.,  subsistence  and  treat- 
ment) and  private  costs  (e.g.,  loss  of  patient  in- 
come). McCaffree  found  that  the  costs  per  pa- 
tient in  intensive  therapy  were  about  50  percent 
less  than  the  costs  of  custodial  care.  Moneys 
were  saved  in  intensive  therapy  primarily 
because  patient  stays  were  much  shorter  (22  v. 
42  days).  Unfortunately,  McCaffree's  data  are 
nonexperimental,  and  some  of  his  data  indicate 
that  the  intensive  therapy  patients  may  have 
been  less  disabled  than  the  custodial  care  group. 
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In  addition,  McCaffree  ignores  psychological 
measures  of  effectiveness.  It  is  possible  that  the 
intensive  therapy  group  was  no  better  after 
receiving  treatment,  but  that  criteria  for  patient 
discharge  had  been  altered  between  the  two  data 
gathering  periods.  Also  important  is  the  fact 
that  not  only  was  more  psychotherapy  offered 
in  intensive  treatment,  but  (see  173)  drug  treat- 
ment was  introduced  at  the  same  time. 

A  later  study  by  May  (173)  assessed  the  cost 
effectiveness  of  milieu  care  (see  also  24)  and  four 
other  treatment  methods.  A  patient  in  milieu 
care  is  provided  a  therapeutic  environment 
(nurses  and  others  who  deal  with  the  patient  are 
specially  trained).  May  used  experimental  pro- 
cedures to  assign  schizophrenic  patients  to  treat- 
ment conditions  and  assessed  both  costs  and  ef- 
fectiveness. He  found  that  milieu  therapy  was 
almost  as  costly  as  psychotherapy,  but  was  the 
least  effective  of  the  five  treatments.  Drugs 
without  psychotherapy  were  the  least  costly 
treatment,  and  drugs  with  psychotherapy  were 
the  most  effective.  It  is  difficult  to  determine 
whether  May's  results  are  generalizable  to  other 
patient  conditions  and  psychotherapy/ chemo- 
therapy as  they  are  currently  provided. 

Community-Based  Versus 
Institutional  Therapy 

Although  it  appears  to  be  more  cost  beneficial 
and  cost  effective  to  provide  inpatients  with  in- 
tensive treatment,  other  alternatives  have  been 
studied.  Thus,  a  major  focus  of  CEA/CBA  stud- 
ies has  been  community-based  approaches  for 
providing  psychotherapy.  Community  care  has 
the  potential  to  provide  major  cost  savings, 
because  such  care  reduces  the  costs  not  directly 
associated  with  treatment  (i.e.,  the  costs  of 
housing,  feeding,  and  supervising  an  institution- 
alized patient  are  higher  than  the  costs  of  the 
same  services  provided  in  the  community). 

Thus,  for  example,  Binner,  Halpern,  and  Pot- 
ter (22)  used  data  from  almost  600  patients 
who  had  received  either  inpatient  or  outpatient 
treatment.  Binner,  et  al.  quantified  intensity  ac- 
cording to  a  factor-analyzed  combination  of 
resource  utilization,  therapeutic  involvement, 
support  from  the  therapeutic  environment,  and 


length  of  stay  (see  247).  Binner,  et  al.  found  that 
less  intensive  treatment  resulted  in  higher  bene- 
fit/cost ratios.  Categorizing  patients  by  severity 
of  dysfunction  at  admission,  and  stratifying 
therapy  intensity  at  four  levels,  Binner,  et  al. 
showed  that  benefit/cost  ratios  were  greater 
than  1  (benefits  exceeded  costs)  for  all  combina- 
tions of  impairment  and  intensity,  but  were  sig- 
nificantly larger  for  low-intensity  therapy  pro- 
vided to  less  impaired  patients.  Average  bene- 
fit/cost ratios  ranged  from  2.28  for  patients 
treated  more  intensively  to  5.23  for  patients 
given  the  least  intensive  therapy.  These  results 
suggest  that  therapy  may  not  be  effective  for 
severely  impaired  populations  and  that  re- 
sources should  be  directed  to  outpatient 
treatment. 

Although  Binner,  et  al.'s  (22)  study  collected 
data  on  outcomes  as  well  as  cost,  it  is  neverthe- 
less difficult  to  generalize  from  it.  One  problem 
is  that  patients  were  not  randomly  assigned  to 
different  levels  of  therapy  intensity.  The  classi- 
fication scheme  for  intensity,  though  mathe- 
matically sophisticated,  cannot  control  for 
possible  differences  in  the  type  of  patient  who 
was  assigned  to  more  versus  less  intensive 
therapy.  It  is  also  not  clear  whether  the  patients 
who  participated  in  community-based  treat- 
ment had  dysfunctions  as  severe  as  those  of  the 
inpatients.  Furthermore,  the  outcome  measure 
used  by  Binner,  et  al.  was  a  gross  measure  based 
on  a  therapist  rating.  The  valuing  of  this  benefit 
was  done  by  assigning  an  arbitrary  dollar 
amount  to  changes  in  therapists'  ratings.  Final- 
ly, Binner,  et  al.  used  operations  costs,  rather 
than  opportunity  values,  and  this  may  have  led 
to  either  an  underestimate  or  overestimate.  It  is 
difficult  to  determine  what  bias  was  incorpo- 
rated and  how  this  relates  to  the  procedure  they 
used  to  value  therapist-rated  changes. 

In  another  study  of  an  outpatient  treatment 
program,  Washburn,  Vanicelli,  Longabaugh, 
and  Sheff  (292)  studied  a  "daycare"  treatment. 
Daycare  is  actually  a  modified  form  of  inpatient 
treatment  where  patients  return  to  their  own 
homes  (or  families)  each  evening.  The  research- 
ers collected  a  variety  of  effectiveness  data,  in- 
cluding checklists  completed  by  patients,  those 
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who  knew  them  well,  and  their  therapists.  Pa- 
tients were  assigned  randomly  to  daycare  or 
standard  inpatient  care.  Analyses  of  effective- 
ness data  collected  at  6-month  intervals  during 
therapy  and  at  a  2-year  followup  found  only  a 
few  significant  differences.  Daycare  patients 
evidenced  somewhat  less  subjective  distress, 
functioned  better  in  the  community,  and  were 
less  of  a  burden  to  their  families  than  patients 
who  received  standard  inpatient  therapy.  Cost 
data  also  collected  at  6-month  intervals,  how- 
ever, showed  significantly  lower  costs  for  day- 
care. The  lower  cost  of  daycare  treatment  seems 
to  be  the  primary  determinant  of  cost  effective- 
ness, although  daycare  also  seems  to  have  im- 
proved effectiveness  to  some  degree.  What  is 
missing  from  Washburn,  et  al.'s  study,  unfor- 
tunately, is  a  no-treatment  or  a  placebo  treat- 
ment condition,  which  would  allow  inferences 
about  how  much  of  the  outcome  is  caused  by 
factors  not  related  to  treatment  procedures  (i.e., 
spontaneous  remission),  or  by  the  patient's 
expectations. 

In  another  comparative  study  of  community 
versus  institutional  treatment,  Murphy  and 
Datel  (192;  see  also  57)  projected  the  costs  and 
benefits  for  52  mentally  ill  and  mentally  re- 
tarded patients  who  were  placed  in  the  commu- 
nity from  State  institutions.  The  costs  of  com- 
munity care  included  housing  and  subsistence, 
as  well  as  the  costs  of  community  treatment. 
Benefits  included  the  cost  savings  of  not  having 
to  provide  institutional  care  and  the  wages/ 
fringe  benefits  received  from  jobs.  Costs  and 
benefits  were  adjusted  for  present  value  (a  0.08 
discount  rate  was  used)  and  inflation.  Murphy 
and  Datel's  results  were  organized  in  terms  of  12 
patient  categories.  Their  findings  indicated  that 
10-year  projected  benefits  exceeded  10-year  pro- 
jected costs  of  community  care,  yielding  bene- 
fit/cost ratios  of  between  0.99  to  11.86.  The 
average  ratio  was  substantially  greater  than  1, 
indicating  that  community  care  was  superior. 

Murphy  and  Datel's  (192)  study,  although  it 
considered  a  broader  array  of  benefits  and  costs 
than  most  CBA  studies  and  used  present  value 
and  inflation  adjustment  procedures,  does  have 
some  methodological  problems.  Although  it 
was  recognized  that  there  are  psychological  ben- 


efits to  patients  and  their  families  and  data  on 
such  things  as  marriage,  normality  of  appear- 
ance, mobility  in  the  community,  and  employ- 
ment were  presented,  these  data  were  not  used 
in  the  benefit/cost  ratios.  Valuing  such  benefits 
is  admittedly  difficult,  but  Murphy  and  Datel's 
analysis,  as  presented,  may  underestimate  ther- 
apy's benefits.  Another  problem  with  this  study 
that  limits  its  validity  is  that  patients  who  re- 
fused community  placement  were  not  included 
in  the  analysis.  These  individuals  were  consid- 
ered treatment  failures  and  excluded  from  the 
analysis.  If  a  significant  number  of  these  pa- 
tients returned  to  the  institution,  costs  of  estab- 
lishing them  in  the  community  ("set-up"  costs) 
and  of  readmitting  them  to  institutions  should 
have  been  incorporated  in  the  CBA.  The  study 
also  didn't  consider  cost  ratios  for  an  alternative 
program.  As  discussed  in  the  preceding  chapter, 
there  always  is  an  implicit  alternative  to  which  a 
cost-benefit  calculation  is  compared  and  not  all 
programs  with  benefit/cost  ratios  exceeding  1 
should  be  continued.  An  obvious  comparison 
group  for  the  Murphy  and  Datel  study  would 
have  been  a  randomly  selected  group  from  the 
potentially  deinstitutionalizable  population  that 
was  retained  in  the  institution  or  was  released  to 
another  program. 

Cassell,  Smith,  Grunberg,  Boan,  and  Thomas 
(32)  obtained  cost  and  effectiveness  data  for 
almost  500  deinstitutionalized  patients,  most  of 
whom  had  been  diagnosed  as  chronic  schizo- 
phrenics. These  patients  had  resided  in  mental 
hospitals  for  an  average  of  18.2  years  prior  to 
deinstitutionalization,  and  the  costs  of  hospitali- 
zation had  totaled  $70,000  per  patient  before 
release.  After  deinstitutionalization,  the  costs  of 
welfare,  followup,  drugs,  and  rehospitalization 
for  those  who  could  not  adapt  to  community 
living  was  only  $1,575  annually,  or  2.44  times 
less  than  the  cost  of  institutionalization.  Al- 
though very  few  of  the  patients  would  have 
been  discharged  had  their  deinstitutionalization 
program  not  begun,  49  percent  of  the  men  and 
38  percent  of  the  women  under  age  65  were  em- 
ployed at  least  3  months  during  the  2-year 
postrelease  period.  More  than  20  percent  of  the 
men  and  10  percent  of  the  women  were  em- 
ployed for  at  least  13  months  following  their 
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release.  Utilization  of  medical  care  by  the  dein- 
stitutionalized patients  was  also  significantly 
less  than  for  normally  adjusted  persons  in  simi- 
lar groups.  In  sum,  the  cost  was  less  and  effec- 
tiveness greater  for  noninstitutional  compared 
to  institutional  care  provided  to  severely  dis- 
turbed patients.  Other  studies  of  deinstitutional- 
ization, which  similarly  have  tracked  patients 
after  release,  have  yielded  similar  findings  (see, 
e.g.,  79,156,191). 

An  important  methodological  limitation  of 
these  deinstitutionalization  studies  is  their  use 
of  nonexperimental  designs.  Typically,  these 
CEA/CBA  studies  follow  deinstitutionalized  pa- 
tients, accounting  for  the  resources  they  require. 
This  is  potentially  problematic  given  the  fact 
that  new  treatments  were  simultaneously  being 
introduced  within  mental  hospitals  (see  173,175) 
and  the  fact  that  patients  who  were  deinstitu- 
tionalized are  likely  to  be  less  disabled  than 
those  who  remain  in  the  hospital.  There  is  also  a 
problem  of  adequately  assessing  the  costs  of 
community  care.  Although  most  studies  include 
major  housing  and  support  costs  for  mainte- 
nance of  patients  in  the  community,  a  compre- 
hensive accounting  of  community  costs  may  be 
difficult  (especially  in  comparison  to  the  ob- 
viousness of  many  costs  borne  in  a  hospital).  As 
a  result  of  these  factors,  the  high  benefit/cost 
ratios  for  community  treatment  may  be  overly 
optimistic. 

The  study  reported  by  Weisbrod  (296;  see 
also  297)  illustrates  the  implications  of  consider- 
ing these  factors  comprehensively  in  a  CBA. 
Weisbrod  compared  traditional  therapy  in  a 
mental  institution  against  a  community  care 
program  using  random  assignment  of  patients. 
Therapy  in  the  mental  institution  was  brief 
(usually  less  than  1  month)  and  was  followed  by 
community  care  delivered  by  local  mental 
health  agencies.  The  usual  stay  in  the  hospital 
was  17  days,  but  many  patients  returned  to  the 
institution.  Therapy  in  the  alternative  commu- 
nity-based program  involved  relatively  little 
time  in  institutions  (i.e.,  every  effort  was  made 
not  to  hospitalize  patients).  During  this  14- 
month  "community  living"  trial,  patients  had  to 
cope  with  normal  living  conditions  and  were  re- 
quired to  take  responsibility  for  the  problems 


that  might,  in  other  circumstances,  result  in 
their  return  to  an  institution. 

Sixty-five  patients  were  included  in  each 
therapy  condition  and  data  were  collected  for 
the  first  12  months  of  treatment.  Weisbrod's 
(296)  cost  assessment  procedures  were  compre- 
hensive and  included  a  variety  of  direct  and  in- 
direct costs  of  treatment.  Included  also  were  law 
enforcement  costs  and  family  burden  costs  (e.g., 
lost  wages  caused  by  the  patient).  Weisbrod's 
results  indicated  that  community  care  was 
slightly  more  costly,  but  yielded  higher  benefit 
than  the  institutional  treatment  program.  The 
costs  were  higher  in  the  community  program  be- 
cause patients  were  closely  supervised  and  re- 
ceived intensive  therapy.  The  benefits  of  com- 
munity treatment  were  higher  primarily  because 
the  outpatients  earned  more  from  employment. 

Despite  the  use  of  random  assignment  and 
comprehensive  assessment  of  cost  and  benefits, 
there  are  important  limitations  to  the  general- 
izability  of  Weisbrod's  (296)  findings.  Although 
data  were  collected  over  a  relatively  long  period 
of  time  (4  years,  from  1972  to  1976),  each  pa- 
tient participated  for  only  14  months.  That  may 
be  too  short  a  time  to  assess  the  effects  of  treat- 
ment. In  addition,  only  a  rather  small  number 
of  patients  were  included  (less  than  150),  all 
from  the  same  geographic  area.  It  is  not  clear 
whether  the  program  actually  studied  a  repre- 
sentative sample  of  those  eligible  for  deinstitu- 
tionalization. There  is  also  no  evidence  that 
either  community  or  institutional  treatment  is 
superior  to  no  treatment,  since  a  no-treatment 
control  condition  was  not  included.  Of  course, 
the  patients  in  Weisbrod's  study  were  severe- 
ly dysfunctional,  and  withholding  treatment 
would  have  been  difficult. 

Residential  Versus  Institutional 
Therapy  for  Problem  Children 

Most  of  the  aforementioned  studies  involved 
schizophrenics  and  other  severely  disturbed 
adults.  Another  typical  application  of  CEA/ 
CBA  in  assessing  mental  health  treatments  are 
studies  which  have  examined  the  use  of  psycho- 
therapy for  "problem  populations"  (see  63).  For 
example,  there  has  been  great  interest  in  study- 
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ing  the  use  of  psychotherapy  with  juvenile  delin- 
quents. There  is  evidence  that  a  substantial 
number  of  serious  crimes  are  committed  by  ju- 
veniles under  14  years  of  age  (see  286),  and  there 
is  considerable  correlational  evidence  that  crim- 
inal behavior  in  youths  is  related  to  adult  crime 
and  mental  dysfunction  (see,  e.g.,  233).  The 
magnitude  of  the  costs  associated  with  these 
problems  is  very  great. 

A  number  of  studies  have  investigated  the  use 
of  behavior  modification  procedures  with  trou- 
bled youths.  Phillips  (213),  for  example,  pre- 
sented data  on  the  cost  and  effectiveness  of  a 
residential  token  economy  therapy  for  juvenile 
delinquents  called  Achievement  Place  (see  also 
243).  Token  economy  therapy  is  an  application 
of  learning  theory  to  establish  a  therapeutic 
milieu.  Along  with  the  direct  costs  of  treatment, 
Phillips  assessed  effectiveness  data  on  several 
variables,  including  police  and  court  contacts, 
school  attendance,  and  grades  before,  during, 
and  after  therapy.  Before  therapy,  Achievement 
Place  youths  averaged  about  3.8  contacts  per 
year  with  the  police  or  courts.  After  1  year  of 
therapy,  Achievement  Place  delinquents  aver- 
aged only  0.75  police  and  court  contacts  and  no 
contacts  2  years  after  therapy.  In  comparison  to 
a  traditional  program  for  juvenile  delinquents  (a 


special  school),  Achievement  Place  was  much 
more  effective  and  one-third  as  costly  ($6,000  v. 
$20,000  to  $30,000  per  delinquent). 

Although  Phillips,  et  al.  (213)  provide  very 
positive  cost-effectiveness  data,  it  should  be 
noted  that  their  findings  are  in  contrast  to  those 
of  Powers  and  Witmer  (217).  In  what  has  be- 
come a  classic  study  (known  as  the  Cambridge- 
Somerville  youth  study),  650  youths  were 
matched  and  randomly  assigned  to  either  long- 
term  counseling  and  supervision  or  no  mental 
health  treatment.  The  participants  in  this  study 
were  tracked  from  1939  to  1976,  and  data  on  a 
variety  of  psychological  and  social  dimensions 
were  collected  (see  also  178).  Unfortunately,  no 
long-term  benefits  can  be  demonstrated,  and 
there  is  some  disturbing  evidence  that  the  ther- 
apy-treated youths  have  faired  worse  than  those 
who  received  no  treatment  (on  such  criteria  as 
alcoholism  and  later  mental  health).  It  is  impor- 
tant to  recognize,  however,  that  the  treatment 
received  by  these  youths  may  be  far  different 
from  that  provided  by  current  standards  (see 
269).  In  some  respects,  the  Cambridge-Somer- 
ville  approach  is  not  comparable  in  rigor  to 
treatments  such  as  Achievement  Place  (which 
was  based  on  a  clear-cut  theoretical  rationale). 


THERAPIST  VARIABLES 

Because  the  major  proportion  of  resources 
used  in  psychotherapy  relates  to  the  cost  of  per- 
sonnel time,  a  number  of  cost  comparisons  of 
different  therapists  have  been  conducted.  In- 
cluded in  these  studies  is  an  analysis  of  the  dif- 
ferences in  fees  charged  by  various  professionals 
and  paraprofessionals  and  the  impact  of  thera- 
pist variables  on  outcome. 


Professional  Therapists 

Karon  and  VandenBos  (132)  report  one  of  the 
few  available  studies  on  the  cost  effectiveness  of 
different  professional  therapists.  A  small  num- 
ber of  patients  diagnosed  as  schizophrenics  were 
treated  by  psychologists,  by  psychiatrists,  or  by 


drugs  without  therapy.  Karon  and  VandenBos 
found  that  the  cost  of  therapy  provided  by  psy- 
chologists was  substantially  less  than  the  cost  of 
therapy  provided  by  psychiatrists,  primarily 
owing  to  lower  hospitalization  and  drug  use  in 
therapy  provided  by  psychologists.  The  average 
cost  per  patient  treated  by  psychologists  was 
$7,813,  but  the  average  cost  per  patient  was 
$12,221  when  psychiatrists  provided  treatment 
and  $17,234  when  drugs  alone  were  used  in 
therapy.  Long-term  costs  due  to  recidivism  also 
favored  psychologists  over  psychiatrists.  Pa- 
tients treated  by  psychologists  were  hospitalized 
an  average  7.2  days  in  a  2-year  followup,  com- 
pared to  93.5  days  for  patients  treated  by  psy- 
chiatrists and  99.8  days  for  patients  treated  by 
drugs  alone. 
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Although  these  findings  suggest  that  psy- 
chologists may  be  more  cost-effective  therapists 
for  schizophrenics  than  psychiatrists,  the  gener- 
alizability  of  these  data  is  hard  to  determine.  Fee 
schedules  for  psychologists  vary  widely,  and  the 
specific  conditions  of  Karon  and  VandenBos'  re- 
search setting  may  have  affected  the  results  in 
unknown  ways.  One  important  outcome  was 
that  therapy  given  by  either  type  of  professional 
was  less  costly  than  drugs  alone:  17  percent  less 
when  provided  by  psychiatrists,  and  33  percent 
less  when  provided  by  psychologists.  While, 
again,  it  is  difficult  to  know  the  generalizability 
of  these  findings,  these  data  suggest  an  area  for 
further  CBAs. 

In  a  related  study,  Karon  and  VandenBos 
(133)  report  that  therapy  provided  by  better 
trained  therapists  is  more  effective,  though 
more  costly,  than  therapy  provided  by  less 
trained  therapists.  The  reduction  in  costs  results 
from  generally  shorter  lengths  of  stay  and  other 
savings  in  types  of  services  required.  This  find- 
ing was  obtained  despite  a  gross  underestimate 
of  the  actual  cost  of  hospitalization  (132).  The 
generalizability  of  such  studies,  however,  may 
be  limited  to  the  type  of  patient  they  used. 

There  is  evidence,  for  example,  that  hospital- 
ized neurotics  can  be  treated  by  nurse  therapists 
with  better  cost/benefit  ratios  than  with  thera- 
pists who  have  more  training  (i.e.,  a  doctorate). 
Ginsberg  and  Marks  (95)  assessed  the  effec- 
tiveness of  treatment  by  nurse  therapists  for 
brief  behavioral  psychotherapy  (patients  used 
an  average  of  9  sessions)  for  neurotic  and 
phobic  patients.  Ginsberg  and  Marks  used  data 
from  1  year  prior  to  therapy  and  1  year  post- 
therapy  to  estimate  the  impact  of  therapy.  The 
data  indicated  that  therapy  significantly  re- 
duced symptomology  and  resulted  in  a  number 
of  tangible  (i.e.,  valued)  benefits.  The  benefits 
included  reduced  use  of  medical  services  and  im- 
proved work  productivity.  Since  the  study  did 
not  employ  comparison  groups,  it  is  impossible 
to  estimate  the  comparative  effects  of  nurse- 
therapists;  however,  the  study  does  suggest  a 
potentially  useful  direction  for  future  analyses. 

Gabby  and  Leavitt  (89)  report  similar  findings 
for  therapy  on  a  population  of  neurotics.  A  self- 
supporting  nonprofit  clinic  provided  long-term 


therapy  and  was  able  to  charge  an  average  of  50 
percent  of  the  fees  charged  by  private  practi- 
tioners in  the  area  ($13.50  to  $15/hour  v. 
$19.95/hour  at  that  time).  A  key  factor  in  the 
clinic's  being  able  to  offer  lower  fees  was  its  ex- 
tensive use  of  nonpsychiatric  staff.  Psychiatrists 
were  called  in  only  when  necessary  for  consulta- 
tion. Although  Gabby  and  Leavitt  provide  an 
example  of  how  costs  can  be  reduced,  they  do 
not  report  outcome  data  in  any  detail,  it  is  un- 
clear whether  the  effects  of  treatment  in  the 
clinic  setting  were  as  good  as  those  in  traditional 
settings.  In  addition,  it  is  unclear  whether  pa- 
tients at  the  clinic  were  more  or  less  disturbed 
than  other  patients  in  the  community.  There  is 
also  some  reason  to  suspect  that  they  attracted  a 
select  population  of  patients. 

Nonprofessionals 

One  focus  of  CEA/CBA  studies  has  been  the 
training  of  parents  to  provide  therapy  to  dis- 
turbed children  (see,  e.g.,  111,211,290).  In  one 
study,  Rubenstein,  Armentrout,  Levin,  and 
Herald  (243)  placed  36  emotionally  disturbed 
children  in  normal  homes.  Parents  residing  in 
these  homes  had  received  training  in  child  man- 
agement skills  and  received  salaries  and  expense 
reimbursement  for  caring  for  the  disturbed 
children.  Lengths  of  stay  ranged  from  9  to  26 
months,  and  the  patients  were  supervised  by 
other  "parent  therapists."  Professional  mental 
health  specialists  also  met  weekly  with  parents. 
Psychometric  tests,  grades,  and  a  behavior 
checklist  showed  that  children  treated  by  parent 
therapists  were  as  improved  as  comparable 
children  treated  in  two  residential  programs. 
Costs  of  parent-therapist  treatment  were  only 
half  of  residential  treatment  costs  ($30.60  v. 
$63.77  per  child  per  day). 

What  seems  to  have  reduced  costs  was  the 
distribution  of  professional  expertise  over  a 
number  of  "helpers."  This  simple  modification 
of  the  traditional  way  of  providing  psychother- 
apy (direct  contact  between  professional  thera- 
pists and  patient)  would  be  expected  to  reduce 
costs.  However,  the  cost  reductions  suggested 
by  the  Rubenstein,  et  al.  (243)  study  probably 
are  overestimated  because  they  do  not  include 
many  of  the  costs  of  training  parents.  The  cost 
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findings  also  are  limited  by  Rubens tein,  et  al.'s 
use  of  "comparable"  rather  than  randomly  se- 
lected comparison  groups.  Perhaps  the  children 
assigned  to  the  parent-therapist  program  were 
less  severely  disturbed  or  had  characteristics 
that  predisposed  them  to  benefit  more  from 
parent  therapy  than  other  children.  As  is  typical 
of  many  CEA  and  CBA  studies,  costs  used  in 
Rubenstein,  et  al.'s  study  were  accounting  costs, 
rather  than  opportunity  costs.  These  costs  may 
well  underestimate  the  use  of  resources. 

Other  CBA  and  CEA  studies  have  explicitly 
compared  the  use  of  professional  versus  para- 
professionals.  Yates  (306),  for  example,  reports 
a  study  of  paraprofessionals  who  conducted 
therapy  for  obese  patients.  The  therapy  was  less 
than  one-tenth  as  costly  as  therapy  provided  by 
psychologists  and  psychiatrists,  but  appeared  to 
be  equally  effective.  Effectiveness  was  measured 
directly  in  terms  of  the  number  of  pounds  lost. 
Yates'  study,  however,  used  a  quasi-experimen- 
tal design  in  which  patients  selected  their  own 
therapy,  and  it  may  be  that  those  who  selected 
the  paraprofessional  treatment  condition  be- 
lieved they  needed  less  help  to  lose  weight  (i.e., 
were  most  motivated).  Also,  while  it  can  be  as- 
sumed that  nonprofessionals  are  less  costly  than 
professionals,  costs  were  not  directly  reported 
(see  also  282). 

Self-directed  therapy,  with  or  without  con- 
sultation with  a  professional,  has  also  begun  to 
receive  some  attention  (see  96).  Many  of  these 
studies  have  focused  on  dysfunctions  typically 


not  treated  by  professional  psychotherapy,  but 
there  are  some  examples  with  depression,  anxie- 
ty, phobias,  and  sexual  dysfunctions.  Such  ther- 
apies involve  the  use  of  a  book  or  manual  to 
direct  the  individual  and  have  been  mainly 
based  on  learning  and  phenomenological  theo- 
ries. Case  studies  of  therapies  directed  by 
manuals  have  yielded  results  indicating  that 
positive  effects  can  be  obtained  with  such  low- 
cost  therapies  (e.g.,  294,299),  but  a  thorough 
evaluation  of  the  effectiveness  and  cost  effec- 
tiveness of  these  procedures  has  not  been  done. 
The  lack  of  comprehensive  cost  data  in  such 
studies  further  limits  any  conclusions  about  cost 
effectiveness  or  cost  benefit. 

One  formal  cost-effectiveness  study  of  self- 
directed  therapy  was  conducted  by  Marston, 
Marston,  and  Ross  (170).  Marston,  et  al.  mailed 
obese  patients  weekly  weight-reduction  read- 
ings ("bibliotherapy")  and  had  them  respond  to 
written  questions  about  the  readings  and 
weight-loss  problems.  They  found  their  form  of 
therapy  to  be  as  effective  as  weekly  visits  with  a 
professional  therapist  and  much  less  costly.  Un- 
fortunately, the  range  of  cost  and  effectiveness 
data  collected  was  limited;  in  addition,  these 
programs  have  not  been  compared  with  tradi- 
tional psychotherapy  treatments.  Further  stud- 
ies have  shown  that  some  contact  with  a  profes- 
sional or  paraprofessional  therapist  is  necessary 
for  these  bibliotherapies  to  be  effective  com- 
pared to  no-treatment  groups,  even  if  this  con- 
tact is  made  by  phone  or  mail  (305,307). 


DRUG  ADDICTION  AND  MEDICAL  UTILIZATION 


The  studies  cited  above  principally  focus  on 
CEA  and  CBA  of  different  ways  of  offering  psy- 
chotherapeutic treatments.  CEA  and  CBA  re- 
ports have  also  been  developed  to  study  treat- 
ments for  problems  such  as  alcohol  and  heroin 
addiction.  These  are  high-cost  problems  to  soci- 
ety and  are  also  problems  that  have  highly  vis- 
ible costs  and  benefits/effects.  Overutilization 
of  medical  services,  also  a  costly  problem,  has 
also  received  considerable  attention  (e.g.,  60, 
129),  and  it  has  a  number  of  implications  for 
how  and  under  what  conditions  psychotherapy 


should  be  evaluated  in  a  national  health  insur- 
ance program.  Below  several  CEA/CBA  studies 
of  treatment  for  these  high-cost  problems  are 
described. 

Psychotherapy  for  Drug  Addictions 

The  obvious  relationship  between  alcohol 
and  opiate  addictions  and  subsequent  psycho- 
logical dysfunction,  coupled  with  the  high 
societal  costs  of  such  addictions,  has  stimulated 
a  number  of  CEA  and  CBA  studies  of  therapies 
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for  drug  addictions.  Involvement  of  multidisci- 
plinary  teams  in  therapy  for  drug  addictions 
seem  to  have  generated  more  comprehensive 
CEA  and  CBA  in  this  area  than  in  any  other 
and,  in  general,  the  findings  of  these  studies  is 
that  therapy  is  beneficial.  This  conclusion  must 
be  tempered,  however,  by  recognizing  that 
treatments  for  drug  addiction  almost  always  in- 
volve more  than  psychotherapy. 

Rufener  and  his  colleagues  (244,245),  for  ex- 
ample, conducted  a  CBA  of  five  different  thera- 
pies for  heroin  addiction:  1)  methadone  mainte- 
nance, 2)  therapeutic  community,  3)  outpatient 
drug-free  treatment,  4)  outpatient  detoxifica- 
tion, and  5)  inpatient  detoxification.  Benefits 
were  calculated  by  estimating  social  costs  in- 
curred directly  and  indirectly  as  a  result  of  her- 
oin use  (e.g.,  crime,  incarceration,  court  costs), 
the  reduction  in  costs  produced  by  rehabilitat- 
ing a  heroin  abuser,  and  adjustment  of  benefits 
for  the  relative  effectiveness  of  different  pro- 
grams. Costs  were  assessed  from  accounting 
records  of  providing  each  therapy.  Rufener,  et 
al.  considered  benefits  under  three  assumptions 
of  the  size  of  heroin  abuser  population  and  three 
different  discount  rates  for  present-valuing.  The 
resulting  ratios  were  all  greater  than  1  (benefits 
exceed  costs)  and  showed  that  outpatient  drug- 
free  therapy  to  be  the  most  cost  beneficial. 
Drug-free  therapy  yielded  a  12.82  benefit/cost 
ratio  for  the  intermediate  abuser  population  size 
(and  included  discount  rate  assumptions).  A 
possible  problem  of  this  has  to  do  with  esti- 
mated adjustment  cost  savings  for  differential 
relapse  rates  which  may  have  been  overly  opti- 
mistic. In  addition,  the  study  did  not  use  ran- 
dom assignment  of  patients  to  different  treat- 
ment techniques,  and  it  is  difficult  to  determine 
the  possible  effects  of  the  research  design. 

In  another  study,  Hall,  Bass,  Hargreaves,  and 
Loeb  (112)  report  a  20-percent  reduction  in  the 
use  of  opiates  and  barbiturates  for  outpatient 
detoxification  patients.  Random  assignment  of 
subjects  to  behavior  therapy  and  no  treatment 
conditions  was  accomplished,  and  effectiveness 
was  assessed  by  urine  tests.  The  behavior  ther- 
apy consisted  of  reinforcing  patients  (by  paying 
them  up  to  $10/day)  for  drug-free  urines.  De- 
pending on  the  day  urines  were  collected,  40  to 


50  percent  of  the  paid  subjects  had  not  used 
drugs.  Since  most  estimates  of  the  daily  cost  to 
society  of  heroin  use  are  five  or  more  times  that 
cost,  this  reinforcement  contingency  seems 
promisingly  cost  beneficial,  even  when  person- 
nel, facility,  and  testing  costs  are  added.  Other 
data  suggest  that  patients  did  not  use  their 
payments  to  purchase  illegal  drugs. 

Sirotnik  and  Bailey  (262),  in  a  similar  study, 
conducted  a  CBA  of  heroin  addiction  therapies. 
Their  investigation  involved  285  patients  over  a 
1.5-year  period.  They  found  that  comprehen- 
sively defined  benefits  exceeded  costs  by  more 
than  a  2.5  ratio  under  each  of  a  range  of  cost- 
savings  assumptions.  This  analysis  probably  is 
conservative,  because  it  did  not  consider  ben- 
efits that  might  accrue  to  patients  and  society 
after  the  program.  Unfortunately,  nonrandom 
assignment  to  therapies  and  the  absence  of  a 
control  group  limit,  the  implications  of  this 
study. 

Even  larger  benefit/cost  ratios  may  be  pro- 
duced by  long-term  therapy  for  addicts,  al- 
though there  is  much  debate  on  this  point.  Aron 
and  Daily  (8),  for  example,  found  that  long- 
term  therapy  was  more  effective  and,  in  total, 
less  costly  than  short-term  therapy  for  drug  ad- 
diction when  costs  of  re-entry  to  therapy  and 
long-run  effectiveness  were  assessed.  Cost-effec- 
tiveness ratios  were  $4,624  per  successfully 
treated  addict  in  long-term  therapy,  but  $5,988 
per  successful  patient  in  short-term  therapy. 

Maidlow  and  Berman  (167)  contrasted  the 
cost  effectiveness  and  cost  benefit  of  a  drug- 
free  therapeutic  community  and  methadone  ap- 
proaches to  treating  heroin  addiction.  The  aver- 
age stay  of  4  years  in  drug-free  communities  was 
found  to  generate  a  65-percent  success  rate, 
resulting  in  a  direct  cost  per  successful  patient  of 
$17,760.  Methadone  or  other  drug  substitution 
had  a  higher,  87-percent  success  rate  at  a  life- 
time cost  effectiveness  of  $45,000  per  successful 
patient.  Considering  the  probability  that  former 
addicts  would  leave  therapy,  and  present-valu- 
ing future  benefits  and  costs  (thus  reducing  life- 
time cost  of  drug  substitution),  it  would  seem 
drug  substitution  is  more  cost  beneficial  than 
drug-free  therapeutic  communities.  However, 
Maidlow  and  Berman's  study,  although  it  uses 
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sound  CBA  methodology,  did  not  use  random 
assignment  or  a  no-treatment  control  group  and 
there  may  be  important  biases  in  who  selects 
particular  treatment. 

Goldschmidt  (101)  described  a  comprehensive 
model  of  CEA  for  health  care,  applying  it  to 
analysis  of  heroin  addiction  therapies.  Sampling 
1,640  patients  over  a  6-month  period  he  found 
1,241  who  could  be  interviewed  and  used  data 
from  them  to  compare  the  cost  effectiveness  of 
drug  substitution  (methadone)  to  the  therapeu- 
tic community.  Using  several  variables  to  assess 
effectiveness  and  operations  cost  for  the  two 
approaches,  high  variability  was  found  in  non- 
recidivism  measures  of  effectiveness  (10.9  to 
33.7  percent  for  methadone  therapy,  depend- 
ing largely  on  length  of  stay,  v.  12.5  to  47  per- 
cent for  therapeutic  communities).  Goldschmidt 
found  that  the  annual  cost  per  patient  of  the 
therapeutic  community  was  about  four  times 
the  cost  of  drug  substitution.  Because  thera- 
peutic communities  were  found  to  treat  more 
patients  and  have  somewhat  more  persistent  ef- 
fects, the  average  cost/successful  patient  ratio 
was  1.7  times  higher  for  therapeutic  commu- 
nities than  for  drug  substitution,  thus  favoring 
drug  substitution  as  more  effective  for  its  cost. 
This  finding  was  maintained  for  both  "nor- 
malcy" (i.e.,  some  use  of  drugs,  but  normal 
functioning)  and  "heroin-free"  criteria  of  effec- 
tiveness. This  study  shows  the  impact  that  treat- 
ment system  costs  can  have  on  CEA  findings. 
Consideration  of  lifetime  costs  of  methadone 
maintenance,  however,  might  reverse  the  direc- 
tion or  magnitude  of  the  findings.  No  informa- 
tion was  provided  on  opportunity  value  and 
comprehensive  costs  and,  as  well,  no-treatment 
control  groups  were  not  evaluated. 

To  summarize  the  status  of  CEA  and  CBA  for 
drug  addictions,  while  this  area  has  received 
much  attention,  there  are  still  serious  short- 
comings with  the  available  research.  These 
shortcomings  have  to  do  primarily  with  the  re- 
search designs  used.  Although  some  of  the  find- 
ings are  impressive  in  that  consistent  cost/bene- 
fit ratios  greater  than  unity  are  found,  there 
may  be  other  explanations  for  these  results.  In 
addition,  it  should  be  recognized  that  drug 
abuse  remains  a  serious  problem  and  it  is  not 


clear  that  the  available  studies  are  representa- 
tive of  the  treatment  programs  being  conducted. 

Psychotherapy  and  Overutilization  of 
Medical  Services 

Depending  on  the  definition  used,  it  has  been 
estimated  that  medical  services  are  overused  (or 
in  some  other  way  abused)  by  between  20  and 
60  percent  of  those  who  seek  them  (e.g.,  54, 
174,182).  Various  researchers  and  analysts  have 
concluded  that  medical  services  often  are  used 
to  ameliorate  mental  problems,  rather  than  the 
treatment  of  physical  dysfunctions.  Regier,  et 
al.'s  (225)  data  on  the  role  of  primary  care 
physicians  in  the  delivery  of  mental  health  serv- 
ices is  one  indication  of  this  problem  (see  ch.  1). 
It  has  been  suggested  that  psychotherapy  may 
be  used  to  reduce  individuals'  dependence  on 
medical  services.  Thus,  for  example,  a  4-year 
study  by  Cavanaugh  (cited  in  119)  found  that 
hospitalization  for  physical  ailments  was  re- 
duced from  an  average  111  to  an  average  53 
days  by  psychotherapy,  resulting  in  a  $1.1  mil- 
lion savings  (which  was  greater  than  psycho- 
therapy costs). 

Cummings  and  Follette  (53;  see  also  99)  re- 
ported that  a  single  session  of  psychotherapy  re- 
duced utilization  of  medical  services  for  high 
utilizer  patients  by  60  percent  over  a  5-year 
period.  Additional  sessions  further  reduced 
medical  care  utilization:  patients  attending  two 
to  eight  sessions  subsequently  decreased  medical 
use  by  75  percent.  Continuing  their  study  on  use 
of  medical  services  in  the  large  Kaiser-Perma- 
nente  health  insurance  plan,  Cummings  (51) 
used  findings  from  Cummings  (50)  and  Cum- 
mings and  Follette  (52,53)  to  contrast  the  bene- 
fits of  medical  cost  savings  to  costs  of  four  dura- 
tions of  psychotherapy. 

Cummings  formed  benefit/cost  ratios  by  di- 
viding medical  care  utilization  (number  of  vis- 
its) for  the  year  preceding  therapy  by  the  sum  of 
subsequent  medical  care  and  psychotherapy  vis- 
its. Very  brief  psychotherapy  (one  to  four  ses- 
sions) generated  "cost-therapeutic  effectiveness" 
(actually  crude  benefit/cost)  ratios  of  2.59.  Psy- 
chotherapy lasting  from  1  to  15  sessions  had  a 
similarly  positive  benefit/cost  ratio  of  2.11. 


Ch.  6 — Illustrative  Cost-Effectiveness  and  Cost-Benefit  Analyses  *  73 


Long-term  therapy  (more  than  16  sessions)  and 
interminable  therapy  were  found  to  be  less  cost- 
beneficial  (benefit/cost  ratios  of  1.14  and  0.91, 
respectively).  Not  receiving  any  psychotherapy, 
however,  generated  the  worst  ratio  of  all:  0.88. 

Other  investigators  have  found  specific  re- 
ductions in  use  of  physician  services,  especially 
laboratory  and  X-ray  services.  In  one  report, 
Goldensohn  and  Fink  (100)  found  that  these  ef- 
fects followed  after  the  administration  of  psy- 
chological tests.  A  number  of  other  studies 
show  similar,  and  often  monetized,  reductions 
in  medical  care  utilization  following  a  variety  of 
forms  of  psychotherapy,  including  brief  treat- 
ment (e.g.,  125).  In  many  of  the  studies,  reduc- 
tions in  medical  utilization  have  been  directly 
related  to  reduced  costs. 

Jones  and  Vischi  (129),  at  the  National  Insti- 
tute of  Mental  Health,  have  conducted  a  com- 
prehensive review  of  the  literature  on  the  "off- 
set" benefits  of  psychotherapy  treatments.  They 
reviewed  25  studies  that  examined  the  impact  of 
therapy  for  mental  illness,  alcohol  abuse,  and 
drug  abuse  on  medical  utilization.  Their  conclu- 
sion was  that,  under  certain  circumstances, 
medical  care  utilization  does  "appear"  to  be 
reduced  as  a  result  of  therapy.  In  particular,  for 
treatments  of  mental  illness,  they  found  12  stud- 
ies (out  of  13  available  studies)  which  demon- 
strated reduced  medical  care  utilization  follow- 
ing therapy.  The  median  reduction  was  20  per- 
cent, and  the  range  (for  the  12  studies)  was  5  to 
85  percent.  The  reduction  in  medical  care  util- 
ization was  accompanied  by  parallel  reduction 


in  cost  (mental  health  treatments  being  typically 
less  costly  than  medical  treatment).  The  investi- 
gators report  a  variety  of  benefit/cost  ratios, 
which  range  from  0.95  to  2.11. 

Jones  and  Vischi  (129)  also  reported  a  variety 
of  studies  in  which  psychotherapeutic  treat- 
ments were  applied  to  alcoholism  and  drug 
abuse.  These  studies  report  similar  findings  (of 
reduced  medical  care  utilization)  although  they 
tended  to  have  more  methodological  problems 
than  the  mental  illness  studies.  A  particular 
problem  for  all  of  the  studies  was  the  frequent 
lack  of  appropriate  control  group  conditions. 
Another  problem  was  the  selection  of  time  peri- 
ods for  analysis.  Typically,  patients  entered 
these  studies  after  a  period  of  high  use  of  serv- 
ices and  it  is  difficult  to  attribute  the  reduction 
solely  to  the  mental  health  treatment  (it  could 
reflect  a  natural  change  in  medical  needs).  Jones 
and  Vischi,  although  optimistic  about  the  use  of 
psychotherapy  to  reduce  medical  utilization, 
recommend  more  rigorous  research  that  in- 
cludes better  cost-benefit  data. 

In  a  number  of  cases,  psychotherapy  has  been 
employed  as  an  adjunct  to  medical  services 
(e.g.,  143,204,215,238).  Although  there  is  not 
yet  a  substantial  literature  describing  the  effec- 
tiveness of  psychotherapeutic  treatments  used  in 
this  way,  such  fields  a$  "behavioral  medicine" 
are  being  developed.  It  would  seem  that  as  the 
effectiveness  literature  on  such  behavioral- 
mental  treatment  matures,  cost  and  benefit  data 
should  also  be  assessed. 


SUMMARY 

This  chapter  has  provided  a  number  of  illus- 
trations of  the  use  of  CEA/CBA  methods  for 
assessing  a  variety  of  psychotherapy  treatment 
alternatives  and  mental  health  problems.  The 
conduct  of  CEA/CBA  studies  of  psychotherapy 
research  is  not  widespread,  and  many  important 
areas  and  types  of  psychotherapy  have  been 
ignored  to  date.  The  review  in  this  chapter  sug- 
gests that,  for  some  treatment  system  character- 
istics and  for  some  problems,  psychotherapy 


appears  to  be  cost  beneficial  and  that  it  can  be 
made  more  efficient  (improved  cost  effective- 
ness). These  findings  must  be  considered  tenu- 
ous, however,  given  the  methodological  prob- 
lems inherent  in  many  of  these  studies.  What  is 
needed  is  a  more  systematic  application  of 
rigorous  research  designs  that  employ  no-treat- 
ment and  placebo  treatment  conditions,  as  well 
as  more  comprehensive  measures  of  outcome 
and  cost. 
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It  would  seem  that  we  possess  the  scientific 
tools  to  assess  psychotherapy  and  to  use  this  in- 
formation in  making  policy  decisions  about 
societal  support  for  these  treatments.  It  would 
also  appear  that  we  have  substantial  knowledge 
which  indicates  that  these  treatments  offer  the 


potential  to  improve  well-being  and  to  be  cost 
effective.  Although  one  will  probably  never  be 
totally  satisfied  with  the  answers  provided  by 
research,  that  seems  a  poor  reason  not  to  at- 
tempt more  rigorous  and  comprehensive  scien- 
tific analyses. 


Appendixes 


Appendix  A.— Description  of 
Other  Volumes  of  the  Assessment 


The  overall  OTA  assessment,  The  Implications  of 
Cost-Effectiveness  Analysis  of  Medical  Technology, 
consists  of  a  main,  policy-oriented  report  plus  five 
background  papers.  The  present  volume,  The  Ef- 
ficacy and  Cost  Effectiveness  of  Psychotherapy,  is 
one  of  the  background  papers.  The  main  report  and 
the  other  background  efforts  are  briefly  described 
below. 

The  main  report  examines  three  major  issues:  1) 
the  general  usefulness  of  CEA/CBA  in  decisionmak- 
ing regarding  medical  technology,  2)  the  methodo- 
logical strengths  and  shortcomings  of  the  technique, 
and  3)  the  potential  for  initiating  or  expanding  the 
use  of  CEA/CBA  in  six  health  care  programs  (reim- 
bursement coverage,  health  planning,  market  ap- 
proval for  drugs  and  medical  devices,  Professional 
Standards  Review  Organizations,  R&D  activities, 
and  health  maintenance  organizations),  and  most  im- 
portantly, the  implications  of  any  expanded  use. 

The  prime  focus  of  the  report  is  on  the  application 
of  CEA/CBA  to  medical  technology  (i.e.,  the  drugs, 
devices,  and  medical  and  surgical  procedures  used  in 
medical  care,  and  the  organizational  and  support  sys- 
tems within  which  such  care  is  provided).  With  the 
exception  of  the  present  background  paper  on  psy- 
chotherapy, the  report  does  not  address  psychosocial 
medicine.  Other  aspects  of  health,  such  as  the  envi- 
ronment, are  not  directly  covered  either.  The  find- 
ings of  the  assessment,  though,  might  very  well  apply 
to  health  care  resource  decisionmaking  in  general, 
and  with  modification,  to  other  policy  areas  such  as 
education,  the  environment,  and  occupational  safety 
and  health. 

The  main  report  contains  chapters  on  methodol- 
ogy, general  decisionmaking,  each  of  the  six  health 
programs  mentioned  above,  and  the  general  useful- 
ness of  CEA/CBA.  It  contains  appendixes  covering  a 
survey  of  current  and  past  uses  of  CEA/CBA  by 
agencies  (primarily  Federal),  a  survey  of  the  resource 
costs  involved  in  conducting  CEA/CBAs,  a  discus- 
sion of  ethical  issues  and  CEA/CBA,  and  a  brief  dis- 
cussion of  legal  issues. 

Background  Paper  #2:  Methodological  Issues  and 
Literature  Review,  includes  an  in-depth  examination 
of  the  decisionmaking  context  and  methodology  dis- 
cussions presented  in  this  report.  A  critique  of  the  lit- 
erature, a  bibliography  of  over  600  items,  and  ab- 
stracts of  over  70  studies  and  other  articles  are  also 
included. 

In  order  to  help  examine  the  applicability  of  tech- 
niques to  assess  the  costs  and  benefits  of  medical 


technology,  19  case  studies  were  prepared.  All  19  are 
available  individually.  In  addition,  17  of  the  cases  are 
available  collectively  in  a  volume  entitled  Back- 
ground Paper  #2:  Case  Studies  of  Medical  Technol- 
ogies. Some  of  the  cases  represent  formal  CEAs  (e.g., 
the  case  on  bone  marrow  transplants),  and  some  rep- 
resent net  cost  or  "least  cost"  analysis  (e.g.,  the  case 
on  certain  respiratory  therapies).  Other  cases  illus- 
trate various  issues  such  as  the  difficulty  of  conduct- 
ing CEA  in  the  absence  of  adequate  efficacy  and  safe- 
ty information  (e.g.,  the  case  on  breast  cancer  sur- 
gery), or  the  role  and  impact  of  formal  analysis  on 
policymaking  (e.g.,  the  case  on  end-stage  renal  dis- 
ease interventions).  The  17  case  studies  in  Back- 
ground Paper  #2  and  their  authors  are: 

Artificial  Heart 

Deborah  P.  Lubeck 

John  P.  Bunker 
Automated  Multichannel  Chemistry  Analyzers 

Milton  C.  Weinstein 

Laurie  A.  Pearlman 
Bone  Marrow  Transplants 

Stuart  O.  Schweitzer 

C.  C.  Scalzi 
Breast  Cancer  Surgery 

Karen  Schachter 

Duncan  Neuhauser 
Cardiac  Radionuclide  Imaging 

William  B.  Stason 

Eric  Fortess 
Cervical  Cancer  Screening 

Bryan  R.  Luce 
Cimetidine  and  Peptic  Ulcer  Disease 

Harvey  V.  Fineberg' 

Laurie  A.  Pearlman 
Colon  Cancer  Screening 

David  M.  Eddy 
CT  Scanning 

Judith  L.  Wagner 
Elective  Hysterectomy 

Carol  Korenbrot 

Ann  B.  Flood 

Michael  Higgins 

Noralou  Roos 

John  P.  Bunker 
End-Stage  Renal  Disease  Interventions 

Richard  A.  Rettig 
Gastrointestinal  Endoscopy 

Jonathan  A.  Showstack 

Steven  A.  Schroeder 
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Neonatal  Intensive  Care 
Peter  Budetti 
Peggy  McManus 
Nancy  Barrand 

Lu  Ann  HeinenNurse  Practitioners 

Lauren  LeRoy 
Orthopedic  Joint  Prosthetic  Implants 

Judith  D.  Bentkover 

Philip  G.  Drew 
Periodontal  Disease  Interventions 

Richard  M.  Scheffler 

Sheldon  Rovin 
Respiratory  Therapy 

Richard  M.  Scheffler 

Morgan  Delaney 

The  18th  case  study  is  the  present  volume,  Back- 
ground Paper  #3:  The  Efficacy  and  Cost  Effectiveness 
of  Psychotherapy.  The  19th  case  study  was  prepared 
by  Judith  Wagner  and  is  published  separately  as 
Background  Paper  #5.-  Assessment  of  Four  Common 
X-Ray  Procedures. 

Background  Paper  #4:  The  Management  of  Health 
Care  Technology  in  Ten  Countries  is  an  analysis  of 
the  policies,  programs,  and  methods,  including  cost- 
effectiveness  and  cost-benefit  techniques,  that  nine 
industrialized  nations  other  than  the  United  States 
use  to  manage  the  effects  of  medical  technology.  The 
experience  of  these  nine  countries  in  managing  medi- 
cal technology  is  compared  to  that  of  the  United 
States.  The  paper  on  the  United  States  and  the  com- 
parative analysis  were  prepared  by  OTA  staff,  as- 


sisted by  Louise  Russell.  The  authors  of  the  papers  on 
the  nine  foreign  countries  are: 
United  Kingdom 

Barbara  Stocking 
Canada 

Jack  Needleman 
Australia 

Sydney  Sax 
Japan 

Joel  Broida 
France 

Rebecca  Fuhrer 
Germany 

Karin  A.  Dumbaugh 
Netherlands 

L.M.J.  Groot 
Iceland 

David  Gunnarson 

Duncan  vB.  Neuhauser 
Sweden 

Erik  H.  G.  Gaensler 

Egon  Jonsson 

Duncan  vB.  Neuhauser 

A  related  report  prepared  by  OTA  and  reviewed 
by  the  Advisory  Panel  to  the  overall  assessment  is  A 
Review  of  Selected  Federal  Vaccine  and  Immuniza- 
tion Policies.  That  study,  published  in  September  of 
1979,  examined  vaccine  research,  development,  and 
production;  vaccine  efficacy,  safety,  and  cost  effec- 
tiveness; liability  issues;  and  factors  affecting  the  use 
of  vaccines.  Pneumococcal  vaccine  was  used  as  a 
case  study,  and  a  CEA/CBA  was  performed. 
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Frederick  C.  Robbins,  Chairman 
Dean,  School  of  Medicine,  Case  Western  Reserve  University 
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Dean 

Florence  Heller  School 
Brandeis  University 

Robert  M.  Ball 
Senior  Scholar 
Institute  of  Medicine 
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The  Office  of  Technology  Assessment  (OTA)  was  created  in  1972 
as  an  advisory  arm  of  Congress.  OTA's  basic  function  is  to  help  legislative 
policymakers  anticipate  and  plan  for  the  consequences  of  technological 
changes  and  to  examine  the  many  ways,  expected  and  unexpected,  in 
which  technology  affects  people's  lives.  The  assessment  of  technology 
calls  for  exploration  of  the  physical,  biological,  economic,  social,  and  po- 
litical impacts  which  can  result  from  applications  of  scientific  knowledge. 
OTA  provides  Congress  with  independent  and  timely  information  about 
the  potential  effects — both  beneficial  and  harmful — of  technological  ap- 
plications. 

Requests  for  studies  are  made  by  chairmen  of  standing  committees 
of  the  House  of  Representatives  or  Senate;  by  the  Technology  Assessment 
Board,  the  governing  body  of  OTA;  or  by  the  Director  of  OTA  in  consul- 
tation with  the  Board. 

The  Technology  Assessment  Board  is  composed  of  six  members  of 
the  House,  six  members  of  the  Senate,  and  the  OTA  Director,  who  is  a 
nonvoting  member. 

OTA  currently  has  underway  studies  in  nine  general  areas —  ener- 
gy; materials;  international  security  and  commerce;  food  and  renewable 
resources;  genetics  and  population;  health;  space;  communication  and  in- 
formation technologies;  oceans;  and  transportation. 
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